Jul 26 17:47:14.577795 ip-10-0-128-209 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 26 17:47:14.577807 ip-10-0-128-209 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 26 17:47:14.577816 ip-10-0-128-209 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 26 17:47:14.578132 ip-10-0-128-209 systemd[1]: Failed to start Kubernetes Kubelet. Jul 26 17:47:23.939080 ip-10-0-128-209 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 26 17:47:23.939100 ip-10-0-128-209 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot ae1202b0ca01411a9923f1b37aead0d3 -- Jul 26 17:49:04.784097 ip-10-0-128-209 systemd[1]: Starting Kubernetes Kubelet... Jul 26 17:49:05.123837 ip-10-0-128-209 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 26 17:49:05.123837 ip-10-0-128-209 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 26 17:49:05.123837 ip-10-0-128-209 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 26 17:49:05.123837 ip-10-0-128-209 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 26 17:49:05.123837 ip-10-0-128-209 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 26 17:49:05.125433 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.125276 2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 26 17:49:05.129358 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129340 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 26 17:49:05.129358 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129357 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 26 17:49:05.129429 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129361 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 26 17:49:05.129429 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129366 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 26 17:49:05.129429 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129369 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 26 17:49:05.129429 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129372 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 26 17:49:05.129429 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129379 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 26 17:49:05.129429 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129382 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 26 17:49:05.129429 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129385 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 26 17:49:05.129429 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129387 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 26 17:49:05.129429 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129390 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 26 17:49:05.129429 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129392 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 26 17:49:05.129429 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129395 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 26 17:49:05.129429 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129398 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 26 17:49:05.129429 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129400 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 26 17:49:05.129429 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129403 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 26 17:49:05.129429 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129405 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 26 17:49:05.129429 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129408 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 26 17:49:05.129429 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129411 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 26 17:49:05.129429 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129414 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 26 17:49:05.129429 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129416 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 26 17:49:05.129429 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129419 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 26 17:49:05.129909 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129422 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 26 17:49:05.129909 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129424 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 26 17:49:05.129909 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129427 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 26 17:49:05.129909 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129429 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 26 17:49:05.129909 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129432 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 26 17:49:05.129909 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129435 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 26 17:49:05.129909 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129438 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 26 17:49:05.129909 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129440 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 26 17:49:05.129909 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129443 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 26 17:49:05.129909 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129446 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 26 17:49:05.129909 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129449 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 26 17:49:05.129909 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129453 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 26 17:49:05.129909 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129457 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 26 17:49:05.129909 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129460 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 26 17:49:05.129909 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129463 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 26 17:49:05.129909 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129465 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 26 17:49:05.129909 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129469 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 26 17:49:05.129909 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129471 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 26 17:49:05.129909 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129474 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 26 17:49:05.129909 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129477 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 26 17:49:05.130401 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129479 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 26 17:49:05.130401 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129481 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 26 17:49:05.130401 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129484 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 26 17:49:05.130401 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129486 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 26 17:49:05.130401 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129489 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 26 17:49:05.130401 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129491 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 26 17:49:05.130401 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129494 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 26 17:49:05.130401 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129496 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 26 17:49:05.130401 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129499 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 26 17:49:05.130401 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129501 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 26 17:49:05.130401 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129503 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 26 17:49:05.130401 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129506 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 26 17:49:05.130401 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129509 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 26 17:49:05.130401 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129512 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 26 17:49:05.130401 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129515 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 26 17:49:05.130401 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129519 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 26 17:49:05.130401 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129521 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 26 17:49:05.130401 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129524 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 26 17:49:05.130401 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129527 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 26 17:49:05.130882 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129530 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 26 17:49:05.130882 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129532 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 26 17:49:05.130882 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129537 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 26 17:49:05.130882 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129540 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 26 17:49:05.130882 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129543 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 26 17:49:05.130882 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129546 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 26 17:49:05.130882 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129549 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 26 17:49:05.130882 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129552 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 26 17:49:05.130882 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129555 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 26 17:49:05.130882 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129558 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 26 17:49:05.130882 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129561 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 26 17:49:05.130882 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129564 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 26 17:49:05.130882 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129566 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 26 17:49:05.130882 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129569 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 26 17:49:05.130882 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129601 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 26 17:49:05.130882 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129606 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 26 17:49:05.130882 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129610 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 26 17:49:05.130882 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129613 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 26 17:49:05.130882 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129615 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 26 17:49:05.131322 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129619 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 26 17:49:05.131322 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129622 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 26 17:49:05.131322 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129624 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 26 17:49:05.131322 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129627 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 26 17:49:05.131322 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129630 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 26 17:49:05.131322 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129633 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 26 17:49:05.131322 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129637 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 26 17:49:05.131322 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129640 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 26 17:49:05.131322 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129642 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 26 17:49:05.131322 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129645 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 26 17:49:05.131322 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129648 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 26 17:49:05.131322 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129651 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 26 17:49:05.131322 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129653 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 26 17:49:05.131322 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129655 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 26 17:49:05.131322 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129658 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 26 17:49:05.131322 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129661 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 26 17:49:05.131322 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129664 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 26 17:49:05.131322 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129666 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 26 17:49:05.131322 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.129669 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 26 17:49:05.131322 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130382 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 26 17:49:05.131869 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130389 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 26 17:49:05.131869 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130392 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 26 17:49:05.131869 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130395 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 26 17:49:05.131869 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130398 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 26 17:49:05.131869 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130401 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 26 17:49:05.131869 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130404 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 26 17:49:05.131869 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130407 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 26 17:49:05.131869 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130409 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 26 17:49:05.131869 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130412 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 26 17:49:05.131869 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130415 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 26 17:49:05.131869 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130418 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 26 17:49:05.131869 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130420 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 26 17:49:05.131869 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130423 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 26 17:49:05.131869 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130425 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 26 17:49:05.131869 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130428 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 26 17:49:05.131869 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130430 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 26 17:49:05.131869 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130433 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 26 17:49:05.131869 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130435 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 26 17:49:05.131869 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130438 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 26 17:49:05.131869 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130441 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 26 17:49:05.132369 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130444 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 26 17:49:05.132369 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130446 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 26 17:49:05.132369 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130449 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 26 17:49:05.132369 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130452 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 26 17:49:05.132369 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130454 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 26 17:49:05.132369 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130457 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 26 17:49:05.132369 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130459 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 26 17:49:05.132369 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130462 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 26 17:49:05.132369 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130464 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 26 17:49:05.132369 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130466 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 26 17:49:05.132369 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130469 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 26 17:49:05.132369 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130472 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 26 17:49:05.132369 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130474 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 26 17:49:05.132369 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130476 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 26 17:49:05.132369 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130479 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 26 17:49:05.132369 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130482 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 26 17:49:05.132369 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130484 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 26 17:49:05.132369 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130487 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 26 17:49:05.132369 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130490 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 26 17:49:05.132369 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130493 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 26 17:49:05.132873 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130497 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 26 17:49:05.132873 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130501 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 26 17:49:05.132873 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130506 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 26 17:49:05.132873 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130509 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 26 17:49:05.132873 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130512 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 26 17:49:05.132873 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130515 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 26 17:49:05.132873 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130518 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 26 17:49:05.132873 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130520 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 26 17:49:05.132873 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130523 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 26 17:49:05.132873 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130526 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 26 17:49:05.132873 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130529 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 26 17:49:05.132873 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130533 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 26 17:49:05.132873 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130537 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 26 17:49:05.132873 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130540 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 26 17:49:05.132873 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130542 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 26 17:49:05.132873 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130545 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 26 17:49:05.132873 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130549 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 26 17:49:05.132873 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130551 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 26 17:49:05.132873 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130554 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 26 17:49:05.133326 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130556 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 26 17:49:05.133326 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130559 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 26 17:49:05.133326 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130561 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 26 17:49:05.133326 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130564 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 26 17:49:05.133326 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130566 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 26 17:49:05.133326 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130569 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 26 17:49:05.133326 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130571 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 26 17:49:05.133326 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130574 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 26 17:49:05.133326 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130576 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 26 17:49:05.133326 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130579 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 26 17:49:05.133326 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130581 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 26 17:49:05.133326 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130584 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 26 17:49:05.133326 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130587 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 26 17:49:05.133326 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130589 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 26 17:49:05.133326 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130592 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 26 17:49:05.133326 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130594 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 26 17:49:05.133326 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130597 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 26 17:49:05.133326 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130599 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 26 17:49:05.133326 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130602 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 26 17:49:05.133326 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130604 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 26 17:49:05.133830 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130606 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 26 17:49:05.133830 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130609 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 26 17:49:05.133830 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130612 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 26 17:49:05.133830 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130615 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 26 17:49:05.133830 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130618 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 26 17:49:05.133830 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130620 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 26 17:49:05.133830 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130623 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 26 17:49:05.133830 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130625 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 26 17:49:05.133830 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130628 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 26 17:49:05.133830 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130630 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 26 17:49:05.133830 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130633 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 26 17:49:05.133830 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130636 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 26 17:49:05.133830 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130639 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 26 17:49:05.133830 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130641 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 26 17:49:05.133830 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130644 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 26 17:49:05.133830 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130646 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 26 17:49:05.133830 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130649 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 26 17:49:05.133830 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130651 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 26 17:49:05.133830 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.130654 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 26 17:49:05.134287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131062 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 26 17:49:05.134287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131067 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 26 17:49:05.134287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131069 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 26 17:49:05.134287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131072 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 26 17:49:05.134287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131075 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 26 17:49:05.134287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131078 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 26 17:49:05.134287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131080 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 26 17:49:05.134287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131083 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 26 17:49:05.134287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131085 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 26 17:49:05.134287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131089 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 26 17:49:05.134287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131091 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 26 17:49:05.134287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131094 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 26 17:49:05.134287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131096 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 26 17:49:05.134287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131099 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 26 17:49:05.134287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131101 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 26 17:49:05.134287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131105 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 26 17:49:05.134287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131111 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 26 17:49:05.134287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131114 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 26 17:49:05.134287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131116 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 26 17:49:05.134287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131119 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 26 17:49:05.134287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131121 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 26 17:49:05.134817 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131124 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 26 17:49:05.134817 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131127 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 26 17:49:05.134817 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131129 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 26 17:49:05.134817 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131132 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 26 17:49:05.134817 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131134 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 26 17:49:05.134817 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131137 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 26 17:49:05.134817 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131139 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 26 17:49:05.134817 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131142 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 26 17:49:05.134817 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131145 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 26 17:49:05.134817 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131147 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 26 17:49:05.134817 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131149 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 26 17:49:05.134817 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131152 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 26 17:49:05.134817 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131154 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 26 17:49:05.134817 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131156 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 26 17:49:05.134817 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131159 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 26 17:49:05.134817 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131161 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 26 17:49:05.134817 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131164 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 26 17:49:05.134817 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131167 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 26 17:49:05.134817 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131169 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 26 17:49:05.134817 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131172 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 26 17:49:05.135297 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131174 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 26 17:49:05.135297 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131176 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 26 17:49:05.135297 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131179 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 26 17:49:05.135297 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131181 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 26 17:49:05.135297 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131184 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 26 17:49:05.135297 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131187 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 26 17:49:05.135297 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131189 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 26 17:49:05.135297 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131195 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 26 17:49:05.135297 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131199 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 26 17:49:05.135297 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131202 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 26 17:49:05.135297 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131205 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 26 17:49:05.135297 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131208 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 26 17:49:05.135297 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131213 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 26 17:49:05.135297 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131216 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 26 17:49:05.135297 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131219 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 26 17:49:05.135297 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131222 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 26 17:49:05.135297 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131224 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 26 17:49:05.135297 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131227 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 26 17:49:05.135297 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131230 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 26 17:49:05.135803 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131233 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 26 17:49:05.135803 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131235 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 26 17:49:05.135803 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131238 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 26 17:49:05.135803 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131240 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 26 17:49:05.135803 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131243 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 26 17:49:05.135803 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131246 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 26 17:49:05.135803 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131249 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 26 17:49:05.135803 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131251 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 26 17:49:05.135803 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131254 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 26 17:49:05.135803 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131256 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 26 17:49:05.135803 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131259 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 26 17:49:05.135803 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131262 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 26 17:49:05.135803 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131264 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 26 17:49:05.135803 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131267 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 26 17:49:05.135803 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131270 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 26 17:49:05.135803 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131272 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 26 17:49:05.135803 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131275 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 26 17:49:05.135803 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131277 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 26 17:49:05.135803 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131280 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 26 17:49:05.135803 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131282 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 26 17:49:05.136298 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131286 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 26 17:49:05.136298 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131289 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 26 17:49:05.136298 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131292 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 26 17:49:05.136298 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131294 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 26 17:49:05.136298 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131298 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 26 17:49:05.136298 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131300 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 26 17:49:05.136298 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131303 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 26 17:49:05.136298 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131305 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 26 17:49:05.136298 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131308 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 26 17:49:05.136298 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131310 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 26 17:49:05.136298 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131313 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 26 17:49:05.136298 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131316 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 26 17:49:05.136298 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131319 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 26 17:49:05.136298 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131323 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 26 17:49:05.136298 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131327 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 26 17:49:05.136298 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131337 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 26 17:49:05.136298 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131341 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 26 17:49:05.136298 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131344 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 26 17:49:05.136298 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.131347 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 26 17:49:05.136298 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131429 2559 flags.go:64] FLAG: --address="0.0.0.0" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131439 2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131449 2559 flags.go:64] FLAG: --anonymous-auth="true" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131455 2559 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131462 2559 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131467 2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131472 2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131476 2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131479 2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131482 2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131486 2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131489 2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131492 2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131497 2559 flags.go:64] FLAG: --cgroup-root="" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131500 2559 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131503 2559 flags.go:64] FLAG: --client-ca-file="" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131506 2559 flags.go:64] FLAG: --cloud-provider="external" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131509 2559 flags.go:64] FLAG: --cluster-dns="[]" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131514 2559 flags.go:64] FLAG: --cluster-domain="" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131517 2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131520 2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131523 2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131527 2559 flags.go:64] FLAG: --container-log-max-files="5" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131530 2559 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 26 17:49:05.136825 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131533 2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131536 2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131540 2559 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131543 2559 flags.go:64] FLAG: --contention-profiling="false" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131546 2559 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131549 2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131553 2559 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131555 2559 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131560 2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131563 2559 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131566 2559 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131568 2559 flags.go:64] FLAG: --enable-load-reader="false" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131571 2559 flags.go:64] FLAG: --enable-server="true" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131575 2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131580 2559 flags.go:64] FLAG: --event-burst="100" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131583 2559 flags.go:64] FLAG: --event-qps="50" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131586 2559 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131589 2559 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131595 2559 flags.go:64] FLAG: --eviction-hard="" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131599 2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131601 2559 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131606 2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131609 2559 flags.go:64] FLAG: --eviction-soft="" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131613 2559 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131616 2559 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 26 17:49:05.137406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131619 2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131621 2559 flags.go:64] FLAG: --experimental-mounter-path="" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131624 2559 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131627 2559 flags.go:64] FLAG: --fail-swap-on="true" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131630 2559 flags.go:64] FLAG: --feature-gates="" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131634 2559 flags.go:64] FLAG: --file-check-frequency="20s" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131638 2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131641 2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131644 2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131647 2559 flags.go:64] FLAG: --healthz-port="10248" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131650 2559 flags.go:64] FLAG: --help="false" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131653 2559 flags.go:64] FLAG: --hostname-override="ip-10-0-128-209.ec2.internal" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131656 2559 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131659 2559 flags.go:64] FLAG: --http-check-frequency="20s" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131662 2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131665 2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131669 2559 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131672 2559 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131675 2559 flags.go:64] FLAG: --image-service-endpoint="" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131678 2559 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131681 2559 flags.go:64] FLAG: --kube-api-burst="100" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131684 2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131688 2559 flags.go:64] FLAG: --kube-api-qps="50" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131690 2559 flags.go:64] FLAG: --kube-reserved="" Jul 26 17:49:05.138035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131693 2559 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131697 2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131700 2559 flags.go:64] FLAG: --kubelet-cgroups="" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131703 2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131707 2559 flags.go:64] FLAG: --lock-file="" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131709 2559 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131712 2559 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131715 2559 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131721 2559 flags.go:64] FLAG: --log-json-split-stream="false" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131724 2559 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131727 2559 flags.go:64] FLAG: --log-text-split-stream="false" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131729 2559 flags.go:64] FLAG: --logging-format="text" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131732 2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131736 2559 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131738 2559 flags.go:64] FLAG: --manifest-url="" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131741 2559 flags.go:64] FLAG: --manifest-url-header="" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131746 2559 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131749 2559 flags.go:64] FLAG: --max-open-files="1000000" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131753 2559 flags.go:64] FLAG: --max-pods="110" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131756 2559 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131759 2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131762 2559 flags.go:64] FLAG: --memory-manager-policy="None" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131765 2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131768 2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131782 2559 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 26 17:49:05.138606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131786 2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131795 2559 flags.go:64] FLAG: --node-status-max-images="50" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131799 2559 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131802 2559 flags.go:64] FLAG: --oom-score-adj="-999" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131805 2559 flags.go:64] FLAG: --pod-cidr="" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131808 2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131814 2559 flags.go:64] FLAG: --pod-manifest-path="" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131817 2559 flags.go:64] FLAG: --pod-max-pids="-1" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131820 2559 flags.go:64] FLAG: --pods-per-core="0" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131826 2559 flags.go:64] FLAG: --port="10250" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131829 2559 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131834 2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0854cbb0c7ec5962a" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131838 2559 flags.go:64] FLAG: --qos-reserved="" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131841 2559 flags.go:64] FLAG: --read-only-port="10255" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131843 2559 flags.go:64] FLAG: --register-node="true" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131846 2559 flags.go:64] FLAG: --register-with-taints="" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131851 2559 flags.go:64] FLAG: --registry-burst="10" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131854 2559 flags.go:64] FLAG: --registry-qps="5" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131857 2559 flags.go:64] FLAG: --reserved-cpus="" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131859 2559 flags.go:64] FLAG: --reserved-memory="" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131863 2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131866 2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131870 2559 flags.go:64] FLAG: --rotate-certificates="false" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131873 2559 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131876 2559 flags.go:64] FLAG: --runonce="false" Jul 26 17:49:05.139224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131879 2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131882 2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131885 2559 flags.go:64] FLAG: --seccomp-default="false" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131888 2559 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131891 2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131894 2559 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131897 2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131900 2559 flags.go:64] FLAG: --storage-driver-password="root" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131903 2559 flags.go:64] FLAG: --storage-driver-secure="false" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131906 2559 flags.go:64] FLAG: --storage-driver-table="stats" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131908 2559 flags.go:64] FLAG: --storage-driver-user="root" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131911 2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131914 2559 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131918 2559 flags.go:64] FLAG: --system-cgroups="" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131921 2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131926 2559 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131930 2559 flags.go:64] FLAG: --tls-cert-file="" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131933 2559 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131938 2559 flags.go:64] FLAG: --tls-min-version="" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131941 2559 flags.go:64] FLAG: --tls-private-key-file="" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131944 2559 flags.go:64] FLAG: --topology-manager-policy="none" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131947 2559 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131950 2559 flags.go:64] FLAG: --topology-manager-scope="container" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131953 2559 flags.go:64] FLAG: --v="2" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131957 2559 flags.go:64] FLAG: --version="false" Jul 26 17:49:05.139895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131961 2559 flags.go:64] FLAG: --vmodule="" Jul 26 17:49:05.140500 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131965 2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 26 17:49:05.140500 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.131968 2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 26 17:49:05.140500 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132087 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 26 17:49:05.140500 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132090 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 26 17:49:05.140500 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132093 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 26 17:49:05.140500 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132095 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 26 17:49:05.140500 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132098 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 26 17:49:05.140500 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132101 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 26 17:49:05.140500 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132104 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 26 17:49:05.140500 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132106 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 26 17:49:05.140500 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132109 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 26 17:49:05.140500 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132111 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 26 17:49:05.140500 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132114 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 26 17:49:05.140500 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132117 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 26 17:49:05.140500 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132119 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 26 17:49:05.140500 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132122 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 26 17:49:05.140500 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132124 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 26 17:49:05.140500 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132127 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 26 17:49:05.140500 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132129 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 26 17:49:05.140989 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132133 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 26 17:49:05.140989 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132138 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 26 17:49:05.140989 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132141 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 26 17:49:05.140989 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132145 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 26 17:49:05.140989 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132148 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 26 17:49:05.140989 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132151 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 26 17:49:05.140989 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132154 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 26 17:49:05.140989 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132157 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 26 17:49:05.140989 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132160 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 26 17:49:05.140989 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132163 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 26 17:49:05.140989 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132166 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 26 17:49:05.140989 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132168 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 26 17:49:05.140989 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132171 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 26 17:49:05.140989 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132174 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 26 17:49:05.140989 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132176 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 26 17:49:05.140989 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132179 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 26 17:49:05.140989 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132182 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 26 17:49:05.140989 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132186 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 26 17:49:05.140989 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132189 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 26 17:49:05.141457 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132192 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 26 17:49:05.141457 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132194 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 26 17:49:05.141457 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132197 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 26 17:49:05.141457 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132200 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 26 17:49:05.141457 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132202 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 26 17:49:05.141457 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132205 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 26 17:49:05.141457 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132207 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 26 17:49:05.141457 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132210 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 26 17:49:05.141457 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132212 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 26 17:49:05.141457 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132215 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 26 17:49:05.141457 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132218 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 26 17:49:05.141457 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132220 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 26 17:49:05.141457 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132223 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 26 17:49:05.141457 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132226 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 26 17:49:05.141457 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132228 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 26 17:49:05.141457 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132231 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 26 17:49:05.141457 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132235 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 26 17:49:05.141457 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132238 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 26 17:49:05.141457 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132241 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 26 17:49:05.141457 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132243 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 26 17:49:05.142037 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132246 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 26 17:49:05.142037 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132249 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 26 17:49:05.142037 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132251 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 26 17:49:05.142037 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132254 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 26 17:49:05.142037 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132256 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 26 17:49:05.142037 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132259 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 26 17:49:05.142037 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132263 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 26 17:49:05.142037 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132266 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 26 17:49:05.142037 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132269 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 26 17:49:05.142037 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132272 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 26 17:49:05.142037 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132275 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 26 17:49:05.142037 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132277 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 26 17:49:05.142037 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132280 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 26 17:49:05.142037 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132283 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 26 17:49:05.142037 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132285 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 26 17:49:05.142037 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132288 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 26 17:49:05.142037 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132290 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 26 17:49:05.142037 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132293 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 26 17:49:05.142037 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132296 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 26 17:49:05.142037 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132298 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 26 17:49:05.142525 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132301 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 26 17:49:05.142525 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132303 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 26 17:49:05.142525 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132306 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 26 17:49:05.142525 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132308 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 26 17:49:05.142525 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132311 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 26 17:49:05.142525 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132314 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 26 17:49:05.142525 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132316 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 26 17:49:05.142525 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132319 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 26 17:49:05.142525 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132323 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 26 17:49:05.142525 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132326 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 26 17:49:05.142525 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132328 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 26 17:49:05.142525 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132331 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 26 17:49:05.142525 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132334 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 26 17:49:05.142525 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132337 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 26 17:49:05.142525 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132340 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 26 17:49:05.142525 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132342 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 26 17:49:05.142525 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132345 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 26 17:49:05.142525 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132347 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 26 17:49:05.142525 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132350 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 26 17:49:05.143011 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132352 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 26 17:49:05.143011 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132355 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 26 17:49:05.143011 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132357 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 26 17:49:05.143011 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.132360 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 26 17:49:05.143011 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.132969 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 26 17:49:05.143011 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.139716 2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 26 17:49:05.143011 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.139737 2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 26 17:49:05.143011 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.140656 2559 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 26 17:49:05.143011 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.140663 2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 26 17:49:05.143011 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140748 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 26 17:49:05.143011 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140754 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 26 17:49:05.143011 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140757 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 26 17:49:05.143011 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140760 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 26 17:49:05.143011 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140762 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 26 17:49:05.143011 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140765 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 26 17:49:05.143398 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140770 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 26 17:49:05.143398 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140786 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 26 17:49:05.143398 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140790 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 26 17:49:05.143398 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140793 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 26 17:49:05.143398 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140796 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 26 17:49:05.143398 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140799 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 26 17:49:05.143398 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140802 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 26 17:49:05.143398 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140805 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 26 17:49:05.143398 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140808 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 26 17:49:05.143398 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140810 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 26 17:49:05.143398 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140813 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 26 17:49:05.143398 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140815 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 26 17:49:05.143398 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140818 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 26 17:49:05.143398 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140820 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 26 17:49:05.143398 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140823 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 26 17:49:05.143398 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140825 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 26 17:49:05.143398 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140828 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 26 17:49:05.143398 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140831 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 26 17:49:05.143398 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140834 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 26 17:49:05.143899 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140837 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 26 17:49:05.143899 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140839 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 26 17:49:05.143899 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140842 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 26 17:49:05.143899 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140844 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 26 17:49:05.143899 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140854 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 26 17:49:05.143899 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140857 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 26 17:49:05.143899 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140859 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 26 17:49:05.143899 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140862 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 26 17:49:05.143899 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140865 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 26 17:49:05.143899 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140868 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 26 17:49:05.143899 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140870 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 26 17:49:05.143899 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140873 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 26 17:49:05.143899 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140875 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 26 17:49:05.143899 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140878 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 26 17:49:05.143899 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140880 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 26 17:49:05.143899 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140883 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 26 17:49:05.143899 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140885 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 26 17:49:05.143899 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140888 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 26 17:49:05.143899 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140891 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 26 17:49:05.144380 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140893 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 26 17:49:05.144380 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140896 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 26 17:49:05.144380 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140898 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 26 17:49:05.144380 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140901 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 26 17:49:05.144380 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140904 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 26 17:49:05.144380 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140906 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 26 17:49:05.144380 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140909 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 26 17:49:05.144380 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140911 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 26 17:49:05.144380 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140914 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 26 17:49:05.144380 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140916 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 26 17:49:05.144380 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140919 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 26 17:49:05.144380 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140922 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 26 17:49:05.144380 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140924 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 26 17:49:05.144380 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140927 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 26 17:49:05.144380 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140929 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 26 17:49:05.144380 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140932 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 26 17:49:05.144380 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140934 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 26 17:49:05.144380 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140938 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 26 17:49:05.144380 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140948 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 26 17:49:05.144380 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140951 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 26 17:49:05.144891 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140953 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 26 17:49:05.144891 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140956 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 26 17:49:05.144891 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140959 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 26 17:49:05.144891 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140962 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 26 17:49:05.144891 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140964 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 26 17:49:05.144891 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140967 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 26 17:49:05.144891 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140969 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 26 17:49:05.144891 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140972 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 26 17:49:05.144891 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140974 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 26 17:49:05.144891 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140977 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 26 17:49:05.144891 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140979 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 26 17:49:05.144891 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140982 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 26 17:49:05.144891 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140984 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 26 17:49:05.144891 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140987 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 26 17:49:05.144891 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140990 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 26 17:49:05.144891 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140993 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 26 17:49:05.144891 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140995 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 26 17:49:05.144891 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.140998 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 26 17:49:05.144891 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141001 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 26 17:49:05.144891 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141004 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 26 17:49:05.145370 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141007 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 26 17:49:05.145370 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141009 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 26 17:49:05.145370 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141012 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 26 17:49:05.145370 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141014 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 26 17:49:05.145370 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141017 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 26 17:49:05.145370 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141019 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 26 17:49:05.145370 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141022 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 26 17:49:05.145370 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141024 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 26 17:49:05.145370 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141027 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 26 17:49:05.145370 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141031 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 26 17:49:05.145370 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141033 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 26 17:49:05.145370 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141043 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 26 17:49:05.145370 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141046 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 26 17:49:05.145370 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141049 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 26 17:49:05.145370 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141052 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 26 17:49:05.145766 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.141057 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 26 17:49:05.145766 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141200 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 26 17:49:05.145766 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141206 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 26 17:49:05.145766 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141209 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 26 17:49:05.145766 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141212 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 26 17:49:05.145766 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141215 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 26 17:49:05.145766 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141218 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 26 17:49:05.145766 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141221 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 26 17:49:05.145766 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141223 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 26 17:49:05.145766 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141226 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 26 17:49:05.145766 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141228 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 26 17:49:05.145766 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141231 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 26 17:49:05.145766 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141233 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 26 17:49:05.145766 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141236 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 26 17:49:05.145766 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141239 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 26 17:49:06.285077 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141241 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 26 17:49:06.285077 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141244 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 26 17:49:06.285077 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141246 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 26 17:49:06.285077 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141249 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 26 17:49:06.285077 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141252 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 26 17:49:06.285077 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141254 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 26 17:49:06.285077 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141257 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 26 17:49:06.285077 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141259 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 26 17:49:06.285077 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141262 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 26 17:49:06.285077 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141264 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 26 17:49:06.285077 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141267 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 26 17:49:06.285077 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141270 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 26 17:49:06.285077 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141273 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 26 17:49:06.285077 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141275 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 26 17:49:06.285077 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141285 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 26 17:49:06.285077 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141288 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 26 17:49:06.285077 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141291 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 26 17:49:06.285077 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141295 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 26 17:49:06.285077 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141298 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 26 17:49:06.285077 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141301 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 26 17:49:06.285877 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141304 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 26 17:49:06.285877 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141307 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 26 17:49:06.285877 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141309 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 26 17:49:06.285877 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141312 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 26 17:49:06.285877 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141315 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 26 17:49:06.285877 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141317 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 26 17:49:06.285877 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141320 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 26 17:49:06.285877 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141323 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 26 17:49:06.285877 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141326 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 26 17:49:06.285877 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141328 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 26 17:49:06.285877 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141331 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 26 17:49:06.285877 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141334 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 26 17:49:06.285877 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141338 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 26 17:49:06.285877 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141341 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 26 17:49:06.285877 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141344 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 26 17:49:06.285877 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141347 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 26 17:49:06.285877 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141349 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 26 17:49:06.285877 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141352 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 26 17:49:06.285877 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141354 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 26 17:49:06.285877 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141357 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 26 17:49:06.286423 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141359 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 26 17:49:06.286423 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141362 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 26 17:49:06.286423 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141365 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 26 17:49:06.286423 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141368 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 26 17:49:06.286423 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141370 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 26 17:49:06.286423 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141373 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 26 17:49:06.286423 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141375 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 26 17:49:06.286423 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141384 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 26 17:49:06.286423 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141386 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 26 17:49:06.286423 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141389 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 26 17:49:06.286423 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141391 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 26 17:49:06.286423 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141394 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 26 17:49:06.286423 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141396 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 26 17:49:06.286423 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141399 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 26 17:49:06.286423 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141401 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 26 17:49:06.286423 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141404 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 26 17:49:06.286423 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141406 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 26 17:49:06.286423 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141409 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 26 17:49:06.286423 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141411 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 26 17:49:06.286423 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141414 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 26 17:49:06.286962 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141416 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 26 17:49:06.286962 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141419 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 26 17:49:06.286962 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141422 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 26 17:49:06.286962 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141424 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 26 17:49:06.286962 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141427 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 26 17:49:06.286962 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141429 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 26 17:49:06.286962 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141431 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 26 17:49:06.286962 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141435 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 26 17:49:06.286962 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141439 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 26 17:49:06.286962 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141442 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 26 17:49:06.286962 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141444 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 26 17:49:06.286962 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141447 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 26 17:49:06.286962 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141450 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 26 17:49:06.286962 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141452 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 26 17:49:06.286962 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141455 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 26 17:49:06.286962 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141458 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 26 17:49:06.286962 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141461 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 26 17:49:06.286962 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141464 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 26 17:49:06.286962 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141467 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 26 17:49:06.287428 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141470 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 26 17:49:06.287428 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141478 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 26 17:49:06.287428 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141481 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 26 17:49:06.287428 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141483 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 26 17:49:06.287428 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141486 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 26 17:49:06.287428 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:05.141488 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 26 17:49:06.287428 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.141493 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 26 17:49:06.287428 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.141662 2559 server.go:962] "Client rotation is on, will bootstrap in background" Jul 26 17:49:06.287428 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.143756 2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 26 17:49:06.287428 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.144941 2559 server.go:1019] "Starting client certificate rotation" Jul 26 17:49:06.287428 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.145043 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 26 17:49:06.287428 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:05.145073 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 26 17:49:06.287428 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.286038 2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 26 17:49:06.292015 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.291990 2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 26 17:49:06.302487 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.302461 2559 log.go:25] "Validated CRI v1 runtime API" Jul 26 17:49:06.304554 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.304536 2559 log.go:25] "Validated CRI v1 image API" Jul 26 17:49:06.305355 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.305332 2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 26 17:49:06.307559 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.307535 2559 fs.go:135] Filesystem UUIDs: map[5a9ca3cf-60dc-4725-8964-664292d43286:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a3961944-9e80-4ecf-a340-5230e8b526a4:/dev/nvme0n1p3] Jul 26 17:49:06.307647 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.307558 2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 26 17:49:06.312011 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.311893 2559 manager.go:216] Machine: {Timestamp:2026-07-26 17:49:06.311394948 +0000 UTC m=+0.264050638 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101851 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23cd456a904b7df2308a8f7282147f SystemUUID:ec23cd45-6a90-4b7d-f230-8a8f7282147f BootID:ae1202b0-ca01-411a-9923-f1b37aead0d3 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4c:8b:2f:31:9b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4c:8b:2f:31:9b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2a:d1:8b:be:dc:a4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 26 17:49:06.312011 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.311996 2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 26 17:49:06.312176 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.312111 2559 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 26 17:49:06.312459 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.312442 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 26 17:49:06.314873 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.314841 2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 26 17:49:06.315058 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.314877 2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-209.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 26 17:49:06.316002 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.315991 2559 topology_manager.go:138] "Creating topology manager with none policy" Jul 26 17:49:06.316036 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.316006 2559 container_manager_linux.go:309] "Creating device plugin manager" Jul 26 17:49:06.316036 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.316013 2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 26 17:49:06.316606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.316597 2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 26 17:49:06.317868 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.317858 2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 26 17:49:06.321959 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.321947 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 26 17:49:06.322071 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.322062 2559 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 26 17:49:06.322104 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.322091 2559 kubelet.go:486] "Attempting to sync node with API server" Jul 26 17:49:06.322104 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.322101 2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 26 17:49:06.322170 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.322116 2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 26 17:49:06.322170 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.322125 2559 kubelet.go:398] "Adding apiserver pod source" Jul 26 17:49:06.322170 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.322134 2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 26 17:49:06.322542 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.322527 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 26 17:49:06.322611 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.322554 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 26 17:49:06.324434 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.324417 2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 26 17:49:06.325888 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.325869 2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 26 17:49:06.325965 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.325910 2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 26 17:49:06.326692 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.326663 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 26 17:49:06.326765 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.326699 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 26 17:49:06.326765 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.326708 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 26 17:49:06.326765 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.326716 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 26 17:49:06.326765 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.326728 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 26 17:49:06.326765 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.326737 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 26 17:49:06.326765 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.326749 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 26 17:49:06.326765 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.326757 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 26 17:49:06.326765 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.326767 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 26 17:49:06.327002 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.326775 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 26 17:49:06.327002 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.326789 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 26 17:49:06.327002 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.326800 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 26 17:49:06.327431 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.327419 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 26 17:49:06.327483 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.327433 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 26 17:49:06.331769 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.331734 2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-209.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 26 17:49:06.331890 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:06.331863 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-209.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 26 17:49:06.331945 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:06.331929 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 26 17:49:06.332711 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.332687 2559 server.go:1268] "Started kubelet" Jul 26 17:49:06.332818 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.332740 2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 26 17:49:06.332872 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.332808 2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 26 17:49:06.332872 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.332866 2559 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 26 17:49:06.333642 ip-10-0-128-209 systemd[1]: Started Kubernetes Kubelet. Jul 26 17:49:06.334247 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.334117 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wcqq4" Jul 26 17:49:06.335082 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.335066 2559 server.go:310] "Adding debug handlers to kubelet server" Jul 26 17:49:06.335155 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.335130 2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 26 17:49:06.337316 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:06.336479 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-209.ec2.internal.18c5e83616eae40a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-209.ec2.internal,UID:ip-10-0-128-209.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-209.ec2.internal,},FirstTimestamp:2026-07-26 17:49:06.332640266 +0000 UTC m=+0.285295945,LastTimestamp:2026-07-26 17:49:06.332640266 +0000 UTC m=+0.285295945,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-209.ec2.internal,}" Jul 26 17:49:06.337408 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:06.337304 2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 26 17:49:06.341042 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.341027 2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 26 17:49:06.341094 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.341051 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 26 17:49:06.341201 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:06.341179 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-209.ec2.internal\" not found" Jul 26 17:49:06.341650 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.341630 2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 26 17:49:06.341783 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.341631 2559 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 26 17:49:06.341783 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.341684 2559 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 26 17:49:06.341888 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.341807 2559 reconstruct.go:97] "Volume reconstruction finished" Jul 26 17:49:06.341888 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.341814 2559 reconciler.go:29] "Reconciler: start to sync state" Jul 26 17:49:06.341888 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.341819 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wcqq4" Jul 26 17:49:06.342024 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.341957 2559 factory.go:55] Registering systemd factory Jul 26 17:49:06.342024 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.341970 2559 factory.go:223] Registration of the systemd container factory successfully Jul 26 17:49:06.342238 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.342216 2559 factory.go:153] Registering CRI-O factory Jul 26 17:49:06.342238 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.342236 2559 factory.go:223] Registration of the crio container factory successfully Jul 26 17:49:06.342371 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.342286 2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 26 17:49:06.342371 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.342313 2559 factory.go:103] Registering Raw factory Jul 26 17:49:06.342371 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.342345 2559 manager.go:1196] Started watching for new ooms in manager Jul 26 17:49:06.343059 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.343043 2559 manager.go:349] Starting recovery of all containers Jul 26 17:49:06.349942 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.349920 2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 26 17:49:06.352561 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:06.352535 2559 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-209.ec2.internal\" not found" node="ip-10-0-128-209.ec2.internal" Jul 26 17:49:06.354728 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.354626 2559 manager.go:354] Recovery completed Jul 26 17:49:06.357615 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:06.357487 2559 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 26 17:49:06.360592 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.360580 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 26 17:49:06.362298 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.362281 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeHasSufficientMemory" Jul 26 17:49:06.362353 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.362317 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeHasNoDiskPressure" Jul 26 17:49:06.362353 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.362331 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeHasSufficientPID" Jul 26 17:49:06.362895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.362880 2559 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 26 17:49:06.362895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.362894 2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 26 17:49:06.362989 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.362910 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 26 17:49:06.362989 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.362884 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 26 17:49:06.363686 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.363657 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeHasSufficientMemory" Jul 26 17:49:06.363745 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.363697 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeHasNoDiskPressure" Jul 26 17:49:06.363745 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.363708 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeHasSufficientPID" Jul 26 17:49:06.364962 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.364950 2559 policy_none.go:49] "None policy: Start" Jul 26 17:49:06.364996 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.364966 2559 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 26 17:49:06.364996 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.364977 2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 26 17:49:06.366951 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.366940 2559 policy_none.go:47] "Start" Jul 26 17:49:06.410074 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.410057 2559 manager.go:337] "Starting Device Plugin manager" Jul 26 17:49:06.410172 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:06.410098 2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 26 17:49:06.410172 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.410111 2559 server.go:87] "Starting device plugin registration server" Jul 26 17:49:06.410420 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.410406 2559 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 26 17:49:06.410478 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.410422 2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 26 17:49:06.410554 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.410535 2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 26 17:49:06.410636 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.410619 2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 26 17:49:06.410636 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.410633 2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 26 17:49:06.411450 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:06.411430 2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 26 17:49:06.411523 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:06.411478 2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-209.ec2.internal\" not found" Jul 26 17:49:06.431164 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.431135 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 26 17:49:06.440286 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.432314 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 26 17:49:06.440286 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.432351 2559 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 26 17:49:06.440286 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.432371 2559 kubelet.go:2448] "Starting kubelet main sync loop" Jul 26 17:49:06.440286 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:06.432404 2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 26 17:49:06.440286 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.434352 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 26 17:49:06.511359 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.511320 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 26 17:49:06.512286 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.512262 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeHasSufficientMemory" Jul 26 17:49:06.512357 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.512302 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeHasNoDiskPressure" Jul 26 17:49:06.512357 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.512321 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeHasSufficientPID" Jul 26 17:49:06.512357 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.512352 2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-209.ec2.internal" Jul 26 17:49:06.519957 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.519912 2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-209.ec2.internal" Jul 26 17:49:06.519957 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:06.519936 2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-209.ec2.internal\": node \"ip-10-0-128-209.ec2.internal\" not found" Jul 26 17:49:06.532976 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.532948 2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-209.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-209.ec2.internal"] Jul 26 17:49:06.533104 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.533011 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 26 17:49:06.534419 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:06.534399 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-209.ec2.internal\" not found" Jul 26 17:49:06.534681 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.534649 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeHasSufficientMemory" Jul 26 17:49:06.534792 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.534696 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeHasNoDiskPressure" Jul 26 17:49:06.534792 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.534708 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeHasSufficientPID" Jul 26 17:49:06.536971 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.536957 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 26 17:49:06.537075 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.537059 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-209.ec2.internal" Jul 26 17:49:06.537131 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.537100 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 26 17:49:06.537704 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.537689 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeHasSufficientMemory" Jul 26 17:49:06.537801 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.537718 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeHasNoDiskPressure" Jul 26 17:49:06.537801 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.537731 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeHasSufficientPID" Jul 26 17:49:06.537801 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.537735 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeHasSufficientMemory" Jul 26 17:49:06.537801 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.537754 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeHasNoDiskPressure" Jul 26 17:49:06.537801 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.537769 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeHasSufficientPID" Jul 26 17:49:06.539954 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.539937 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-209.ec2.internal" Jul 26 17:49:06.540053 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.539967 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 26 17:49:06.540638 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.540626 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeHasSufficientMemory" Jul 26 17:49:06.540726 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.540650 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeHasNoDiskPressure" Jul 26 17:49:06.540726 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.540663 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeHasSufficientPID" Jul 26 17:49:06.542201 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.542184 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c79721d966a6282ec9f6039747948682-config\") pod \"kube-apiserver-proxy-ip-10-0-128-209.ec2.internal\" (UID: \"c79721d966a6282ec9f6039747948682\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-209.ec2.internal" Jul 26 17:49:06.542286 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.542208 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f64ddc5d2c114aec5edbd37b23d5007-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-209.ec2.internal\" (UID: \"9f64ddc5d2c114aec5edbd37b23d5007\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-209.ec2.internal" Jul 26 17:49:06.542286 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.542237 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f64ddc5d2c114aec5edbd37b23d5007-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-209.ec2.internal\" (UID: \"9f64ddc5d2c114aec5edbd37b23d5007\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-209.ec2.internal" Jul 26 17:49:06.573558 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:06.573532 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-209.ec2.internal\" not found" node="ip-10-0-128-209.ec2.internal" Jul 26 17:49:06.578486 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:06.578468 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-209.ec2.internal\" not found" node="ip-10-0-128-209.ec2.internal" Jul 26 17:49:06.634524 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:06.634494 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-209.ec2.internal\" not found" Jul 26 17:49:06.642816 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.642795 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f64ddc5d2c114aec5edbd37b23d5007-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-209.ec2.internal\" (UID: \"9f64ddc5d2c114aec5edbd37b23d5007\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-209.ec2.internal" Jul 26 17:49:06.642904 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.642819 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f64ddc5d2c114aec5edbd37b23d5007-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-209.ec2.internal\" (UID: \"9f64ddc5d2c114aec5edbd37b23d5007\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-209.ec2.internal" Jul 26 17:49:06.642904 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.642838 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c79721d966a6282ec9f6039747948682-config\") pod \"kube-apiserver-proxy-ip-10-0-128-209.ec2.internal\" (UID: \"c79721d966a6282ec9f6039747948682\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-209.ec2.internal" Jul 26 17:49:06.642904 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.642865 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c79721d966a6282ec9f6039747948682-config\") pod \"kube-apiserver-proxy-ip-10-0-128-209.ec2.internal\" (UID: \"c79721d966a6282ec9f6039747948682\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-209.ec2.internal" Jul 26 17:49:06.642904 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.642888 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f64ddc5d2c114aec5edbd37b23d5007-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-209.ec2.internal\" (UID: \"9f64ddc5d2c114aec5edbd37b23d5007\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-209.ec2.internal" Jul 26 17:49:06.643046 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.642906 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f64ddc5d2c114aec5edbd37b23d5007-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-209.ec2.internal\" (UID: \"9f64ddc5d2c114aec5edbd37b23d5007\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-209.ec2.internal" Jul 26 17:49:06.734860 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:06.734813 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-209.ec2.internal\" not found" Jul 26 17:49:06.835602 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:06.835523 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-209.ec2.internal\" not found" Jul 26 17:49:06.876016 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.875978 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-209.ec2.internal" Jul 26 17:49:06.882448 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:06.882429 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-209.ec2.internal" Jul 26 17:49:06.936498 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:06.936453 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-209.ec2.internal\" not found" Jul 26 17:49:07.037031 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:07.036989 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-209.ec2.internal\" not found" Jul 26 17:49:07.137559 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:07.137474 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-209.ec2.internal\" not found" Jul 26 17:49:07.238097 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:07.238062 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-209.ec2.internal\" not found" Jul 26 17:49:07.283517 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:07.283491 2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 26 17:49:07.284030 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:07.283639 2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 26 17:49:07.284030 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:07.283663 2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 26 17:49:07.338468 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:07.338434 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-209.ec2.internal\" not found" Jul 26 17:49:07.347686 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:07.347627 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-25 17:44:06 +0000 UTC" deadline="2028-02-22 19:21:53.444013815 +0000 UTC" Jul 26 17:49:07.347686 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:07.347663 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13825h32m46.09635463s" Jul 26 17:49:07.411946 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:07.411883 2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 26 17:49:07.438990 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:07.438960 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-209.ec2.internal\" not found" Jul 26 17:49:07.461858 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:07.461819 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f64ddc5d2c114aec5edbd37b23d5007.slice/crio-e5f3a8862e0dd0d2aec55a1f1bccf76d2fc0451fa519bcd24723861bbd998aac WatchSource:0}: Error finding container e5f3a8862e0dd0d2aec55a1f1bccf76d2fc0451fa519bcd24723861bbd998aac: Status 404 returned error can't find the container with id e5f3a8862e0dd0d2aec55a1f1bccf76d2fc0451fa519bcd24723861bbd998aac Jul 26 17:49:07.462398 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:07.462374 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc79721d966a6282ec9f6039747948682.slice/crio-5df81e3d4ca371d268f88d8fd066e9a5f419b1d855b8899062e39f7d37d3281f WatchSource:0}: Error finding container 5df81e3d4ca371d268f88d8fd066e9a5f419b1d855b8899062e39f7d37d3281f: Status 404 returned error can't find the container with id 5df81e3d4ca371d268f88d8fd066e9a5f419b1d855b8899062e39f7d37d3281f Jul 26 17:49:07.466708 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:07.466689 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 26 17:49:07.539574 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:07.539537 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-209.ec2.internal\" not found" Jul 26 17:49:07.588281 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:07.588248 2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 26 17:49:07.641298 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:07.641272 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-209.ec2.internal" Jul 26 17:49:07.651424 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:07.651401 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 26 17:49:07.652246 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:07.652231 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-209.ec2.internal" Jul 26 17:49:07.661720 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:07.661701 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 26 17:49:08.323430 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.323387 2559 apiserver.go:52] "Watching apiserver" Jul 26 17:49:08.331281 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.331101 2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 26 17:49:08.332918 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.332880 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-5cpxq","openshift-network-operator/iptables-alerter-msmbj","openshift-ovn-kubernetes/ovnkube-node-dc9sh","kube-system/global-pull-secret-syncer-fgwwx","openshift-cluster-node-tuning-operator/tuned-226rc","openshift-dns/node-resolver-2575c","openshift-image-registry/node-ca-bh5vf","openshift-multus/multus-xnd6k","openshift-multus/network-metrics-daemon-v5q7r","openshift-network-diagnostics/network-check-target-2sjzw","kube-system/konnectivity-agent-4sdbx","kube-system/kube-apiserver-proxy-ip-10-0-128-209.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-209.ec2.internal"] Jul 26 17:49:08.335823 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.335777 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bh5vf" Jul 26 17:49:08.337993 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.337935 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.338520 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.338497 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 26 17:49:08.338637 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.338617 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cp27q\"" Jul 26 17:49:08.338732 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.338644 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 26 17:49:08.338732 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.338688 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 26 17:49:08.340828 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.340521 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 26 17:49:08.341023 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.341002 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 26 17:49:08.341183 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.341146 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2x284\"" Jul 26 17:49:08.341263 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.341219 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 26 17:49:08.341335 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.341280 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 26 17:49:08.341544 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.341525 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.343632 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.343558 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:08.343632 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:08.343621 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgwwx" podUID="92dc3c1b-48ce-4b80-858c-5421f0bcc2f2" Jul 26 17:49:08.344134 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.344096 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 26 17:49:08.344134 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.344108 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ggpm7\"" Jul 26 17:49:08.344134 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.344119 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 26 17:49:08.344134 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.344131 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 26 17:49:08.344399 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.344372 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 26 17:49:08.344938 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.344909 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 26 17:49:08.344938 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.344911 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 26 17:49:08.345898 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.345877 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.348496 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.348479 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 26 17:49:08.348496 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.348488 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 26 17:49:08.348638 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.348622 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2575c" Jul 26 17:49:08.349062 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.349047 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cn228\"" Jul 26 17:49:08.351043 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.351017 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:08.351138 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.351055 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 26 17:49:08.351138 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:08.351097 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5q7r" podUID="b6d9352c-2170-418f-9836-89640e78d1e1" Jul 26 17:49:08.351242 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.351214 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 26 17:49:08.351293 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.351239 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-d66l6\"" Jul 26 17:49:08.352060 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.352038 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/569a96e6-4c60-43e1-8419-59b3992e3d85-kubelet-dir\") pod \"aws-ebs-csi-driver-node-74r2h\" (UID: \"569a96e6-4c60-43e1-8419-59b3992e3d85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.352141 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.352071 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/569a96e6-4c60-43e1-8419-59b3992e3d85-socket-dir\") pod \"aws-ebs-csi-driver-node-74r2h\" (UID: \"569a96e6-4c60-43e1-8419-59b3992e3d85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.352417 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.352096 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/569a96e6-4c60-43e1-8419-59b3992e3d85-registration-dir\") pod \"aws-ebs-csi-driver-node-74r2h\" (UID: \"569a96e6-4c60-43e1-8419-59b3992e3d85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.353076 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.352646 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-run-openvswitch\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.353076 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.352865 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-host-cni-netd\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.353076 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.352933 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/42f7b319-86ae-4c55-adac-ab5c2a44ed68-ovn-node-metrics-cert\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.353076 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.352969 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/42f7b319-86ae-4c55-adac-ab5c2a44ed68-ovnkube-script-lib\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.353317 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.353236 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/42f7b319-86ae-4c55-adac-ab5c2a44ed68-env-overrides\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.353317 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.353284 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/569a96e6-4c60-43e1-8419-59b3992e3d85-etc-selinux\") pod \"aws-ebs-csi-driver-node-74r2h\" (UID: \"569a96e6-4c60-43e1-8419-59b3992e3d85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.353421 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.353327 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-host-cni-bin\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.353421 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.353362 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-dbus\") pod \"global-pull-secret-syncer-fgwwx\" (UID: \"92dc3c1b-48ce-4b80-858c-5421f0bcc2f2\") " pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:08.353421 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.353389 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/569a96e6-4c60-43e1-8419-59b3992e3d85-device-dir\") pod \"aws-ebs-csi-driver-node-74r2h\" (UID: \"569a96e6-4c60-43e1-8419-59b3992e3d85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.353550 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.353515 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ss5k\" (UniqueName: \"kubernetes.io/projected/569a96e6-4c60-43e1-8419-59b3992e3d85-kube-api-access-4ss5k\") pod \"aws-ebs-csi-driver-node-74r2h\" (UID: \"569a96e6-4c60-43e1-8419-59b3992e3d85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.353594 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.353571 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-host-kubelet\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.353644 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.353603 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-host-slash\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.354644 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.353652 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.354644 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.353805 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret\") pod \"global-pull-secret-syncer-fgwwx\" (UID: \"92dc3c1b-48ce-4b80-858c-5421f0bcc2f2\") " pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:08.354644 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.354238 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-systemd-units\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.354644 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.354302 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-node-log\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.354644 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.354341 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-log-socket\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.354644 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.354530 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/42f7b319-86ae-4c55-adac-ab5c2a44ed68-ovnkube-config\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.354644 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.354596 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq68c\" (UniqueName: \"kubernetes.io/projected/42f7b319-86ae-4c55-adac-ab5c2a44ed68-kube-api-access-jq68c\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.355028 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.354700 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a5ef698f-5999-4902-9fe6-57b185586841-host\") pod \"node-ca-bh5vf\" (UID: \"a5ef698f-5999-4902-9fe6-57b185586841\") " pod="openshift-image-registry/node-ca-bh5vf" Jul 26 17:49:08.355028 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.354759 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-host-run-netns\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.355028 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.354792 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-run-systemd\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.355028 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.354819 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-var-lib-openvswitch\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.355028 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.354861 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-etc-openvswitch\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.355028 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.354890 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a5ef698f-5999-4902-9fe6-57b185586841-serviceca\") pod \"node-ca-bh5vf\" (UID: \"a5ef698f-5999-4902-9fe6-57b185586841\") " pod="openshift-image-registry/node-ca-bh5vf" Jul 26 17:49:08.355028 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.354920 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/569a96e6-4c60-43e1-8419-59b3992e3d85-sys-fs\") pod \"aws-ebs-csi-driver-node-74r2h\" (UID: \"569a96e6-4c60-43e1-8419-59b3992e3d85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.355028 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.354978 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-host-run-ovn-kubernetes\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.355028 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.355004 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-kubelet-config\") pod \"global-pull-secret-syncer-fgwwx\" (UID: \"92dc3c1b-48ce-4b80-858c-5421f0bcc2f2\") " pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:08.355419 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.355038 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-run-ovn\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.355419 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.355067 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wbxv\" (UniqueName: \"kubernetes.io/projected/a5ef698f-5999-4902-9fe6-57b185586841-kube-api-access-2wbxv\") pod \"node-ca-bh5vf\" (UID: \"a5ef698f-5999-4902-9fe6-57b185586841\") " pod="openshift-image-registry/node-ca-bh5vf" Jul 26 17:49:08.356476 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.356315 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 26 17:49:08.357794 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.357777 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.359818 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.359797 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 26 17:49:08.359909 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.359851 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 26 17:49:08.360511 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.360144 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 26 17:49:08.360511 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.360244 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-msmbj" Jul 26 17:49:08.360511 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.360303 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 26 17:49:08.360511 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.360431 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 26 17:49:08.360760 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.360727 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5szc2\"" Jul 26 17:49:08.362527 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.362507 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 26 17:49:08.362860 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.362627 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 26 17:49:08.362860 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.362763 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ffzbq\"" Jul 26 17:49:08.362860 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.362770 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:08.362860 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.362835 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 26 17:49:08.362860 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:08.362844 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2sjzw" podUID="bab84b92-d4aa-422d-a364-195365d32cfb" Jul 26 17:49:08.365113 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.365085 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4sdbx" Jul 26 17:49:08.367356 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.367336 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wfdmr\"" Jul 26 17:49:08.367356 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.367353 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 26 17:49:08.367496 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.367406 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 26 17:49:08.367627 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.367608 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.369783 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.369764 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-22gg5\"" Jul 26 17:49:08.369872 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.369770 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 26 17:49:08.379131 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.379112 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7mgqr" Jul 26 17:49:08.384501 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.384481 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7mgqr" Jul 26 17:49:08.437663 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.437602 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-209.ec2.internal" event={"ID":"c79721d966a6282ec9f6039747948682","Type":"ContainerStarted","Data":"5df81e3d4ca371d268f88d8fd066e9a5f419b1d855b8899062e39f7d37d3281f"} Jul 26 17:49:08.438938 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.438908 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-209.ec2.internal" event={"ID":"9f64ddc5d2c114aec5edbd37b23d5007","Type":"ContainerStarted","Data":"e5f3a8862e0dd0d2aec55a1f1bccf76d2fc0451fa519bcd24723861bbd998aac"} Jul 26 17:49:08.442890 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.442872 2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 26 17:49:08.456492 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456462 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/569a96e6-4c60-43e1-8419-59b3992e3d85-socket-dir\") pod \"aws-ebs-csi-driver-node-74r2h\" (UID: \"569a96e6-4c60-43e1-8419-59b3992e3d85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.456642 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456502 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/569a96e6-4c60-43e1-8419-59b3992e3d85-registration-dir\") pod \"aws-ebs-csi-driver-node-74r2h\" (UID: \"569a96e6-4c60-43e1-8419-59b3992e3d85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.456642 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456527 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/42f7b319-86ae-4c55-adac-ab5c2a44ed68-ovn-node-metrics-cert\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.456642 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456550 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/42f7b319-86ae-4c55-adac-ab5c2a44ed68-ovnkube-script-lib\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.456642 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456575 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-cnibin\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.456642 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456596 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-lib-modules\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.456642 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456619 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2skmr\" (UniqueName: \"kubernetes.io/projected/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-kube-api-access-2skmr\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.456966 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456644 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/42f7b319-86ae-4c55-adac-ab5c2a44ed68-env-overrides\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.456966 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456684 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-host-var-lib-cni-bin\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.456966 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456708 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0afb128f-7308-461d-b7d4-6fae2fcce66f-etc-tuned\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.456966 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456731 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcnld\" (UniqueName: \"kubernetes.io/projected/0afb128f-7308-461d-b7d4-6fae2fcce66f-kube-api-access-kcnld\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.456966 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456744 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/569a96e6-4c60-43e1-8419-59b3992e3d85-socket-dir\") pod \"aws-ebs-csi-driver-node-74r2h\" (UID: \"569a96e6-4c60-43e1-8419-59b3992e3d85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.456966 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456756 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-multus-cni-dir\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.456966 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456778 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be73bb50-4e02-4441-9b09-da833761aac5-cni-binary-copy\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.456966 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456801 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-host-run-k8s-cni-cncf-io\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.456966 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456825 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-multus-conf-dir\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.456966 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456849 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-etc-modprobe-d\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.456966 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456872 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-etc-sysconfig\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.456966 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456895 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trx5l\" (UniqueName: \"kubernetes.io/projected/23a42634-d5de-4d67-a10a-2f5d8872486c-kube-api-access-trx5l\") pod \"iptables-alerter-msmbj\" (UID: \"23a42634-d5de-4d67-a10a-2f5d8872486c\") " pod="openshift-network-operator/iptables-alerter-msmbj" Jul 26 17:49:08.456966 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456922 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/569a96e6-4c60-43e1-8419-59b3992e3d85-device-dir\") pod \"aws-ebs-csi-driver-node-74r2h\" (UID: \"569a96e6-4c60-43e1-8419-59b3992e3d85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.456966 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456948 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-host-kubelet\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.456966 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456937 2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 26 17:49:08.457619 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.456991 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.457619 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457168 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-host-kubelet\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.457619 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457185 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/569a96e6-4c60-43e1-8419-59b3992e3d85-registration-dir\") pod \"aws-ebs-csi-driver-node-74r2h\" (UID: \"569a96e6-4c60-43e1-8419-59b3992e3d85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.457619 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457191 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.457619 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457235 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/569a96e6-4c60-43e1-8419-59b3992e3d85-device-dir\") pod \"aws-ebs-csi-driver-node-74r2h\" (UID: \"569a96e6-4c60-43e1-8419-59b3992e3d85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.457619 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457240 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret\") pod \"global-pull-secret-syncer-fgwwx\" (UID: \"92dc3c1b-48ce-4b80-858c-5421f0bcc2f2\") " pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:08.457619 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457278 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/be73bb50-4e02-4441-9b09-da833761aac5-multus-daemon-config\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.457619 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457283 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/42f7b319-86ae-4c55-adac-ab5c2a44ed68-env-overrides\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.457619 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:08.457380 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 26 17:49:08.457619 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457414 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.457619 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:08.457476 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret podName:92dc3c1b-48ce-4b80-858c-5421f0bcc2f2 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:08.957441092 +0000 UTC m=+2.910096806 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret") pod "global-pull-secret-syncer-fgwwx" (UID: "92dc3c1b-48ce-4b80-858c-5421f0bcc2f2") : object "kube-system"/"original-pull-secret" not registered Jul 26 17:49:08.457619 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457501 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/23a42634-d5de-4d67-a10a-2f5d8872486c-iptables-alerter-script\") pod \"iptables-alerter-msmbj\" (UID: \"23a42634-d5de-4d67-a10a-2f5d8872486c\") " pod="openshift-network-operator/iptables-alerter-msmbj" Jul 26 17:49:08.457619 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457528 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-systemd-units\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.457619 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457545 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq68c\" (UniqueName: \"kubernetes.io/projected/42f7b319-86ae-4c55-adac-ab5c2a44ed68-kube-api-access-jq68c\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.457619 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457563 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a5ef698f-5999-4902-9fe6-57b185586841-host\") pod \"node-ca-bh5vf\" (UID: \"a5ef698f-5999-4902-9fe6-57b185586841\") " pod="openshift-image-registry/node-ca-bh5vf" Jul 26 17:49:08.457619 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457566 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/42f7b319-86ae-4c55-adac-ab5c2a44ed68-ovnkube-script-lib\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.458382 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457590 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tvq6\" (UniqueName: \"kubernetes.io/projected/e0cf3a10-8a54-45ad-aafa-bb7175208585-kube-api-access-4tvq6\") pod \"node-resolver-2575c\" (UID: \"e0cf3a10-8a54-45ad-aafa-bb7175208585\") " pod="openshift-dns/node-resolver-2575c" Jul 26 17:49:08.458382 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457602 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-systemd-units\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.458382 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457615 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-etc-kubernetes\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.458382 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457638 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt8bx\" (UniqueName: \"kubernetes.io/projected/be73bb50-4e02-4441-9b09-da833761aac5-kube-api-access-wt8bx\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.458382 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457651 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a5ef698f-5999-4902-9fe6-57b185586841-host\") pod \"node-ca-bh5vf\" (UID: \"a5ef698f-5999-4902-9fe6-57b185586841\") " pod="openshift-image-registry/node-ca-bh5vf" Jul 26 17:49:08.458382 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457659 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-host\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.458382 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457712 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-os-release\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.458382 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457743 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-host-run-netns\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.458382 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457767 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-run-systemd\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.458382 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457800 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-cni-binary-copy\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.458382 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457819 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-system-cni-dir\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.458382 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457844 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.458382 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457854 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-run-systemd\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.458382 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457867 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5wz7\" (UniqueName: \"kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7\") pod \"network-check-target-2sjzw\" (UID: \"bab84b92-d4aa-422d-a364-195365d32cfb\") " pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:08.458382 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457818 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-host-run-netns\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.458382 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457899 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-run-ovn\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.458382 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457925 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-cnibin\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.459200 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457950 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/569a96e6-4c60-43e1-8419-59b3992e3d85-kubelet-dir\") pod \"aws-ebs-csi-driver-node-74r2h\" (UID: \"569a96e6-4c60-43e1-8419-59b3992e3d85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.459200 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457951 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-run-ovn\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.459200 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457976 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-run-openvswitch\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.459200 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.457996 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-host-cni-netd\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.459200 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458019 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0cf3a10-8a54-45ad-aafa-bb7175208585-tmp-dir\") pod \"node-resolver-2575c\" (UID: \"e0cf3a10-8a54-45ad-aafa-bb7175208585\") " pod="openshift-dns/node-resolver-2575c" Jul 26 17:49:08.459200 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458040 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-host-run-netns\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.459200 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458057 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-host-var-lib-cni-multus\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.459200 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458071 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-etc-sysctl-conf\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.459200 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458084 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-sys\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.459200 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458088 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-host-cni-netd\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.459200 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458093 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-run-openvswitch\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.459200 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458104 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5c23228c-55bb-49e3-b76f-d22199c6a153-agent-certs\") pod \"konnectivity-agent-4sdbx\" (UID: \"5c23228c-55bb-49e3-b76f-d22199c6a153\") " pod="kube-system/konnectivity-agent-4sdbx" Jul 26 17:49:08.459200 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458055 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/569a96e6-4c60-43e1-8419-59b3992e3d85-kubelet-dir\") pod \"aws-ebs-csi-driver-node-74r2h\" (UID: \"569a96e6-4c60-43e1-8419-59b3992e3d85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.459200 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458146 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-host-var-lib-kubelet\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.459200 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458171 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-etc-sysctl-d\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.459200 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458194 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.459200 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458217 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/569a96e6-4c60-43e1-8419-59b3992e3d85-etc-selinux\") pod \"aws-ebs-csi-driver-node-74r2h\" (UID: \"569a96e6-4c60-43e1-8419-59b3992e3d85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.459840 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458260 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-host-cni-bin\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.459840 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458282 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-dbus\") pod \"global-pull-secret-syncer-fgwwx\" (UID: \"92dc3c1b-48ce-4b80-858c-5421f0bcc2f2\") " pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:08.459840 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458296 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-hostroot\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.459840 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458309 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-run\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.459840 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458331 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ss5k\" (UniqueName: \"kubernetes.io/projected/569a96e6-4c60-43e1-8419-59b3992e3d85-kube-api-access-4ss5k\") pod \"aws-ebs-csi-driver-node-74r2h\" (UID: \"569a96e6-4c60-43e1-8419-59b3992e3d85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.459840 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458354 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-host-slash\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.459840 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458357 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/569a96e6-4c60-43e1-8419-59b3992e3d85-etc-selinux\") pod \"aws-ebs-csi-driver-node-74r2h\" (UID: \"569a96e6-4c60-43e1-8419-59b3992e3d85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.459840 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458371 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs\") pod \"network-metrics-daemon-v5q7r\" (UID: \"b6d9352c-2170-418f-9836-89640e78d1e1\") " pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:08.459840 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458386 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23a42634-d5de-4d67-a10a-2f5d8872486c-host-slash\") pod \"iptables-alerter-msmbj\" (UID: \"23a42634-d5de-4d67-a10a-2f5d8872486c\") " pod="openshift-network-operator/iptables-alerter-msmbj" Jul 26 17:49:08.459840 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458404 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-host-cni-bin\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.459840 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458413 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-node-log\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.459840 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458444 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-log-socket\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.459840 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458468 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/42f7b319-86ae-4c55-adac-ab5c2a44ed68-ovnkube-config\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.459840 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458484 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-host-slash\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.459840 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458496 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5c23228c-55bb-49e3-b76f-d22199c6a153-konnectivity-ca\") pod \"konnectivity-agent-4sdbx\" (UID: \"5c23228c-55bb-49e3-b76f-d22199c6a153\") " pod="kube-system/konnectivity-agent-4sdbx" Jul 26 17:49:08.459840 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458522 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-multus-socket-dir-parent\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.459840 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458535 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-node-log\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.460384 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458574 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-log-socket\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.460384 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458613 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-dbus\") pod \"global-pull-secret-syncer-fgwwx\" (UID: \"92dc3c1b-48ce-4b80-858c-5421f0bcc2f2\") " pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:08.460384 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458640 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-var-lib-kubelet\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.460384 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458681 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0afb128f-7308-461d-b7d4-6fae2fcce66f-tmp\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.460384 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458722 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-var-lib-openvswitch\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.460384 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458745 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-etc-openvswitch\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.460384 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458765 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a5ef698f-5999-4902-9fe6-57b185586841-serviceca\") pod \"node-ca-bh5vf\" (UID: \"a5ef698f-5999-4902-9fe6-57b185586841\") " pod="openshift-image-registry/node-ca-bh5vf" Jul 26 17:49:08.460384 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458781 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-system-cni-dir\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.460384 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458795 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-etc-kubernetes\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.460384 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458852 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-etc-systemd\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.460384 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458858 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-var-lib-openvswitch\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.460384 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458889 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/569a96e6-4c60-43e1-8419-59b3992e3d85-sys-fs\") pod \"aws-ebs-csi-driver-node-74r2h\" (UID: \"569a96e6-4c60-43e1-8419-59b3992e3d85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.460384 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458917 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-host-run-ovn-kubernetes\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.460384 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458928 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-etc-openvswitch\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.460384 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458966 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/569a96e6-4c60-43e1-8419-59b3992e3d85-sys-fs\") pod \"aws-ebs-csi-driver-node-74r2h\" (UID: \"569a96e6-4c60-43e1-8419-59b3992e3d85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.460384 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.458985 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-kubelet-config\") pod \"global-pull-secret-syncer-fgwwx\" (UID: \"92dc3c1b-48ce-4b80-858c-5421f0bcc2f2\") " pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:08.460384 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.459007 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42f7b319-86ae-4c55-adac-ab5c2a44ed68-host-run-ovn-kubernetes\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.461032 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.459060 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e0cf3a10-8a54-45ad-aafa-bb7175208585-hosts-file\") pod \"node-resolver-2575c\" (UID: \"e0cf3a10-8a54-45ad-aafa-bb7175208585\") " pod="openshift-dns/node-resolver-2575c" Jul 26 17:49:08.461032 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.459092 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-host-run-multus-certs\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.461032 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.459093 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-kubelet-config\") pod \"global-pull-secret-syncer-fgwwx\" (UID: \"92dc3c1b-48ce-4b80-858c-5421f0bcc2f2\") " pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:08.461032 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.459130 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s657q\" (UniqueName: \"kubernetes.io/projected/b6d9352c-2170-418f-9836-89640e78d1e1-kube-api-access-s657q\") pod \"network-metrics-daemon-v5q7r\" (UID: \"b6d9352c-2170-418f-9836-89640e78d1e1\") " pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:08.461032 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.459171 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wbxv\" (UniqueName: \"kubernetes.io/projected/a5ef698f-5999-4902-9fe6-57b185586841-kube-api-access-2wbxv\") pod \"node-ca-bh5vf\" (UID: \"a5ef698f-5999-4902-9fe6-57b185586841\") " pod="openshift-image-registry/node-ca-bh5vf" Jul 26 17:49:08.461032 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.459199 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-os-release\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.461032 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.459215 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a5ef698f-5999-4902-9fe6-57b185586841-serviceca\") pod \"node-ca-bh5vf\" (UID: \"a5ef698f-5999-4902-9fe6-57b185586841\") " pod="openshift-image-registry/node-ca-bh5vf" Jul 26 17:49:08.461032 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.459423 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/42f7b319-86ae-4c55-adac-ab5c2a44ed68-ovnkube-config\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.461032 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.460847 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/42f7b319-86ae-4c55-adac-ab5c2a44ed68-ovn-node-metrics-cert\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.465968 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.465942 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq68c\" (UniqueName: \"kubernetes.io/projected/42f7b319-86ae-4c55-adac-ab5c2a44ed68-kube-api-access-jq68c\") pod \"ovnkube-node-dc9sh\" (UID: \"42f7b319-86ae-4c55-adac-ab5c2a44ed68\") " pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.469596 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.469568 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ss5k\" (UniqueName: \"kubernetes.io/projected/569a96e6-4c60-43e1-8419-59b3992e3d85-kube-api-access-4ss5k\") pod \"aws-ebs-csi-driver-node-74r2h\" (UID: \"569a96e6-4c60-43e1-8419-59b3992e3d85\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.469823 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.469807 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wbxv\" (UniqueName: \"kubernetes.io/projected/a5ef698f-5999-4902-9fe6-57b185586841-kube-api-access-2wbxv\") pod \"node-ca-bh5vf\" (UID: \"a5ef698f-5999-4902-9fe6-57b185586841\") " pod="openshift-image-registry/node-ca-bh5vf" Jul 26 17:49:08.542870 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.542839 2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 26 17:49:08.560372 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560340 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-host-var-lib-cni-bin\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.560557 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560380 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0afb128f-7308-461d-b7d4-6fae2fcce66f-etc-tuned\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.560557 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560407 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcnld\" (UniqueName: \"kubernetes.io/projected/0afb128f-7308-461d-b7d4-6fae2fcce66f-kube-api-access-kcnld\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.560557 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560434 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-multus-cni-dir\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.560557 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560453 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-host-var-lib-cni-bin\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.560557 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560467 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be73bb50-4e02-4441-9b09-da833761aac5-cni-binary-copy\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.560557 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560502 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-host-run-k8s-cni-cncf-io\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.560557 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560531 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-multus-conf-dir\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.560557 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560555 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-etc-modprobe-d\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.560957 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560577 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-multus-cni-dir\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.560957 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560597 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-etc-sysconfig\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.560957 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560614 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-host-run-k8s-cni-cncf-io\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.560957 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560628 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-multus-conf-dir\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.560957 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560636 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trx5l\" (UniqueName: \"kubernetes.io/projected/23a42634-d5de-4d67-a10a-2f5d8872486c-kube-api-access-trx5l\") pod \"iptables-alerter-msmbj\" (UID: \"23a42634-d5de-4d67-a10a-2f5d8872486c\") " pod="openshift-network-operator/iptables-alerter-msmbj" Jul 26 17:49:08.560957 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560690 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-etc-sysconfig\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.560957 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560694 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/be73bb50-4e02-4441-9b09-da833761aac5-multus-daemon-config\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.560957 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560725 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.560957 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560763 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-etc-modprobe-d\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.560957 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560766 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/23a42634-d5de-4d67-a10a-2f5d8872486c-iptables-alerter-script\") pod \"iptables-alerter-msmbj\" (UID: \"23a42634-d5de-4d67-a10a-2f5d8872486c\") " pod="openshift-network-operator/iptables-alerter-msmbj" Jul 26 17:49:08.560957 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560816 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tvq6\" (UniqueName: \"kubernetes.io/projected/e0cf3a10-8a54-45ad-aafa-bb7175208585-kube-api-access-4tvq6\") pod \"node-resolver-2575c\" (UID: \"e0cf3a10-8a54-45ad-aafa-bb7175208585\") " pod="openshift-dns/node-resolver-2575c" Jul 26 17:49:08.560957 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560841 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-etc-kubernetes\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.560957 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560866 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt8bx\" (UniqueName: \"kubernetes.io/projected/be73bb50-4e02-4441-9b09-da833761aac5-kube-api-access-wt8bx\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.560957 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560888 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-host\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.560957 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560910 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-os-release\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.560957 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.560938 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-cni-binary-copy\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.561706 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561008 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-host\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.561706 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561027 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-os-release\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.561706 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561071 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-etc-kubernetes\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.561706 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561172 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-system-cni-dir\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.561706 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561223 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.561706 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561253 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5wz7\" (UniqueName: \"kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7\") pod \"network-check-target-2sjzw\" (UID: \"bab84b92-d4aa-422d-a364-195365d32cfb\") " pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:08.561706 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561257 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/23a42634-d5de-4d67-a10a-2f5d8872486c-iptables-alerter-script\") pod \"iptables-alerter-msmbj\" (UID: \"23a42634-d5de-4d67-a10a-2f5d8872486c\") " pod="openshift-network-operator/iptables-alerter-msmbj" Jul 26 17:49:08.561706 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561279 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-system-cni-dir\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.561706 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561257 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/be73bb50-4e02-4441-9b09-da833761aac5-multus-daemon-config\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.561706 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561306 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-cnibin\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.561706 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561336 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0cf3a10-8a54-45ad-aafa-bb7175208585-tmp-dir\") pod \"node-resolver-2575c\" (UID: \"e0cf3a10-8a54-45ad-aafa-bb7175208585\") " pod="openshift-dns/node-resolver-2575c" Jul 26 17:49:08.561706 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561361 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-host-run-netns\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.561706 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561386 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-host-var-lib-cni-multus\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.561706 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561400 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-cnibin\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.561706 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561408 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-etc-sysctl-conf\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.561706 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561433 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-sys\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.561706 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561485 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-host-var-lib-cni-multus\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.562526 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561490 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-host-run-netns\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.562526 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561525 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5c23228c-55bb-49e3-b76f-d22199c6a153-agent-certs\") pod \"konnectivity-agent-4sdbx\" (UID: \"5c23228c-55bb-49e3-b76f-d22199c6a153\") " pod="kube-system/konnectivity-agent-4sdbx" Jul 26 17:49:08.562526 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561567 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-host-var-lib-kubelet\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.562526 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561590 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-etc-sysctl-d\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.562526 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561599 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.562526 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561611 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-etc-sysctl-conf\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.562526 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561488 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-sys\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.562526 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561631 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-host-var-lib-kubelet\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.562526 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561630 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-cni-binary-copy\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.562526 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561659 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.562526 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561714 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-hostroot\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.562526 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561733 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-etc-sysctl-d\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.562526 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561739 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-run\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.562526 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561735 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.562526 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561775 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs\") pod \"network-metrics-daemon-v5q7r\" (UID: \"b6d9352c-2170-418f-9836-89640e78d1e1\") " pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:08.562526 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561781 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-run\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.562526 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561789 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-hostroot\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.562526 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561801 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23a42634-d5de-4d67-a10a-2f5d8872486c-host-slash\") pod \"iptables-alerter-msmbj\" (UID: \"23a42634-d5de-4d67-a10a-2f5d8872486c\") " pod="openshift-network-operator/iptables-alerter-msmbj" Jul 26 17:49:08.563247 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561822 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.563247 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561828 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0cf3a10-8a54-45ad-aafa-bb7175208585-tmp-dir\") pod \"node-resolver-2575c\" (UID: \"e0cf3a10-8a54-45ad-aafa-bb7175208585\") " pod="openshift-dns/node-resolver-2575c" Jul 26 17:49:08.563247 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561830 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5c23228c-55bb-49e3-b76f-d22199c6a153-konnectivity-ca\") pod \"konnectivity-agent-4sdbx\" (UID: \"5c23228c-55bb-49e3-b76f-d22199c6a153\") " pod="kube-system/konnectivity-agent-4sdbx" Jul 26 17:49:08.563247 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561859 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-multus-socket-dir-parent\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.563247 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561867 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23a42634-d5de-4d67-a10a-2f5d8872486c-host-slash\") pod \"iptables-alerter-msmbj\" (UID: \"23a42634-d5de-4d67-a10a-2f5d8872486c\") " pod="openshift-network-operator/iptables-alerter-msmbj" Jul 26 17:49:08.563247 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:08.561860 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 17:49:08.563247 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561885 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-var-lib-kubelet\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.563247 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561901 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-multus-socket-dir-parent\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.563247 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561879 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be73bb50-4e02-4441-9b09-da833761aac5-cni-binary-copy\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.563247 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561917 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0afb128f-7308-461d-b7d4-6fae2fcce66f-tmp\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.563247 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:08.561956 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs podName:b6d9352c-2170-418f-9836-89640e78d1e1 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:09.061933309 +0000 UTC m=+3.014588982 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs") pod "network-metrics-daemon-v5q7r" (UID: "b6d9352c-2170-418f-9836-89640e78d1e1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 17:49:08.563247 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.561955 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-var-lib-kubelet\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.563247 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.562018 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-system-cni-dir\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.563247 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.562045 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-etc-kubernetes\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.563247 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.562067 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-etc-systemd\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.563247 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.562096 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e0cf3a10-8a54-45ad-aafa-bb7175208585-hosts-file\") pod \"node-resolver-2575c\" (UID: \"e0cf3a10-8a54-45ad-aafa-bb7175208585\") " pod="openshift-dns/node-resolver-2575c" Jul 26 17:49:08.563247 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.562114 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-system-cni-dir\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.563844 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.562118 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-host-run-multus-certs\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.563844 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.562166 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-etc-systemd\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.563844 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.562175 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e0cf3a10-8a54-45ad-aafa-bb7175208585-hosts-file\") pod \"node-resolver-2575c\" (UID: \"e0cf3a10-8a54-45ad-aafa-bb7175208585\") " pod="openshift-dns/node-resolver-2575c" Jul 26 17:49:08.563844 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.562214 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-etc-kubernetes\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.563844 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.562230 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-host-run-multus-certs\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.563844 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.562165 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s657q\" (UniqueName: \"kubernetes.io/projected/b6d9352c-2170-418f-9836-89640e78d1e1-kube-api-access-s657q\") pod \"network-metrics-daemon-v5q7r\" (UID: \"b6d9352c-2170-418f-9836-89640e78d1e1\") " pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:08.563844 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.562273 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-os-release\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.563844 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.562304 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-cnibin\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.563844 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.562328 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-lib-modules\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.563844 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.562356 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2skmr\" (UniqueName: \"kubernetes.io/projected/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-kube-api-access-2skmr\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.563844 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.562404 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-cnibin\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.563844 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.562422 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5c23228c-55bb-49e3-b76f-d22199c6a153-konnectivity-ca\") pod \"konnectivity-agent-4sdbx\" (UID: \"5c23228c-55bb-49e3-b76f-d22199c6a153\") " pod="kube-system/konnectivity-agent-4sdbx" Jul 26 17:49:08.563844 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.562479 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be73bb50-4e02-4441-9b09-da833761aac5-os-release\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.563844 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.562544 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0afb128f-7308-461d-b7d4-6fae2fcce66f-lib-modules\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.563844 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.563192 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0afb128f-7308-461d-b7d4-6fae2fcce66f-etc-tuned\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.564374 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.564100 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0afb128f-7308-461d-b7d4-6fae2fcce66f-tmp\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.564374 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.564242 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5c23228c-55bb-49e3-b76f-d22199c6a153-agent-certs\") pod \"konnectivity-agent-4sdbx\" (UID: \"5c23228c-55bb-49e3-b76f-d22199c6a153\") " pod="kube-system/konnectivity-agent-4sdbx" Jul 26 17:49:08.568633 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:08.568511 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 26 17:49:08.568633 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:08.568538 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 26 17:49:08.568633 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:08.568551 2559 projected.go:196] Error preparing data for projected volume kube-api-access-x5wz7 for pod openshift-network-diagnostics/network-check-target-2sjzw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 17:49:08.568633 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:08.568608 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7 podName:bab84b92-d4aa-422d-a364-195365d32cfb nodeName:}" failed. No retries permitted until 2026-07-26 17:49:09.06859243 +0000 UTC m=+3.021248114 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-x5wz7" (UniqueName: "kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7") pod "network-check-target-2sjzw" (UID: "bab84b92-d4aa-422d-a364-195365d32cfb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 17:49:08.570741 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.570714 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s657q\" (UniqueName: \"kubernetes.io/projected/b6d9352c-2170-418f-9836-89640e78d1e1-kube-api-access-s657q\") pod \"network-metrics-daemon-v5q7r\" (UID: \"b6d9352c-2170-418f-9836-89640e78d1e1\") " pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:08.571166 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.571134 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt8bx\" (UniqueName: \"kubernetes.io/projected/be73bb50-4e02-4441-9b09-da833761aac5-kube-api-access-wt8bx\") pod \"multus-xnd6k\" (UID: \"be73bb50-4e02-4441-9b09-da833761aac5\") " pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.571295 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.571176 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trx5l\" (UniqueName: \"kubernetes.io/projected/23a42634-d5de-4d67-a10a-2f5d8872486c-kube-api-access-trx5l\") pod \"iptables-alerter-msmbj\" (UID: \"23a42634-d5de-4d67-a10a-2f5d8872486c\") " pod="openshift-network-operator/iptables-alerter-msmbj" Jul 26 17:49:08.571295 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.571181 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcnld\" (UniqueName: \"kubernetes.io/projected/0afb128f-7308-461d-b7d4-6fae2fcce66f-kube-api-access-kcnld\") pod \"tuned-226rc\" (UID: \"0afb128f-7308-461d-b7d4-6fae2fcce66f\") " pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.571758 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.571734 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2skmr\" (UniqueName: \"kubernetes.io/projected/0c0e8dd9-a868-44f9-bec5-aa0aaae878d0-kube-api-access-2skmr\") pod \"multus-additional-cni-plugins-5cpxq\" (UID: \"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0\") " pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.571864 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.571788 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tvq6\" (UniqueName: \"kubernetes.io/projected/e0cf3a10-8a54-45ad-aafa-bb7175208585-kube-api-access-4tvq6\") pod \"node-resolver-2575c\" (UID: \"e0cf3a10-8a54-45ad-aafa-bb7175208585\") " pod="openshift-dns/node-resolver-2575c" Jul 26 17:49:08.647363 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.647293 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bh5vf" Jul 26 17:49:08.656780 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.656755 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" Jul 26 17:49:08.663100 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.663075 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:08.668850 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.668829 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-226rc" Jul 26 17:49:08.675496 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.675479 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2575c" Jul 26 17:49:08.681262 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.681234 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5cpxq" Jul 26 17:49:08.688038 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.688019 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-msmbj" Jul 26 17:49:08.693623 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.693602 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4sdbx" Jul 26 17:49:08.700294 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.700262 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xnd6k" Jul 26 17:49:08.829300 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.829263 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 26 17:49:08.965214 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:08.965140 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret\") pod \"global-pull-secret-syncer-fgwwx\" (UID: \"92dc3c1b-48ce-4b80-858c-5421f0bcc2f2\") " pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:08.965341 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:08.965269 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 26 17:49:08.965341 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:08.965324 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret podName:92dc3c1b-48ce-4b80-858c-5421f0bcc2f2 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:09.965309626 +0000 UTC m=+3.917965292 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret") pod "global-pull-secret-syncer-fgwwx" (UID: "92dc3c1b-48ce-4b80-858c-5421f0bcc2f2") : object "kube-system"/"original-pull-secret" not registered Jul 26 17:49:09.065489 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:09.065460 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs\") pod \"network-metrics-daemon-v5q7r\" (UID: \"b6d9352c-2170-418f-9836-89640e78d1e1\") " pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:09.065649 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:09.065563 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 17:49:09.065649 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:09.065620 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs podName:b6d9352c-2170-418f-9836-89640e78d1e1 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:10.065597526 +0000 UTC m=+4.018253192 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs") pod "network-metrics-daemon-v5q7r" (UID: "b6d9352c-2170-418f-9836-89640e78d1e1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 17:49:09.166565 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:09.166529 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5wz7\" (UniqueName: \"kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7\") pod \"network-check-target-2sjzw\" (UID: \"bab84b92-d4aa-422d-a364-195365d32cfb\") " pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:09.166769 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:09.166722 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 26 17:49:09.166769 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:09.166772 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 26 17:49:09.166944 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:09.166791 2559 projected.go:196] Error preparing data for projected volume kube-api-access-x5wz7 for pod openshift-network-diagnostics/network-check-target-2sjzw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 17:49:09.166944 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:09.166869 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7 podName:bab84b92-d4aa-422d-a364-195365d32cfb nodeName:}" failed. No retries permitted until 2026-07-26 17:49:10.166848071 +0000 UTC m=+4.119503753 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-x5wz7" (UniqueName: "kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7") pod "network-check-target-2sjzw" (UID: "bab84b92-d4aa-422d-a364-195365d32cfb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 17:49:09.321543 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:09.321515 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c23228c_55bb_49e3_b76f_d22199c6a153.slice/crio-e444a40f4911b91092596c29425820e6de174d8f3c440fa2b881cebbcf725c55 WatchSource:0}: Error finding container e444a40f4911b91092596c29425820e6de174d8f3c440fa2b881cebbcf725c55: Status 404 returned error can't find the container with id e444a40f4911b91092596c29425820e6de174d8f3c440fa2b881cebbcf725c55 Jul 26 17:49:09.342095 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:09.342070 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5ef698f_5999_4902_9fe6_57b185586841.slice/crio-4a3c33806987cab72782b4e2c8e25345f1333b8e846e622cb6f949b0db7b06b6 WatchSource:0}: Error finding container 4a3c33806987cab72782b4e2c8e25345f1333b8e846e622cb6f949b0db7b06b6: Status 404 returned error can't find the container with id 4a3c33806987cab72782b4e2c8e25345f1333b8e846e622cb6f949b0db7b06b6 Jul 26 17:49:09.348331 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:09.348308 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0afb128f_7308_461d_b7d4_6fae2fcce66f.slice/crio-f457b3dad9e218d4f8d60601c18e200bba9d7e13ad5e87a1fa4198cb277d6760 WatchSource:0}: Error finding container f457b3dad9e218d4f8d60601c18e200bba9d7e13ad5e87a1fa4198cb277d6760: Status 404 returned error can't find the container with id f457b3dad9e218d4f8d60601c18e200bba9d7e13ad5e87a1fa4198cb277d6760 Jul 26 17:49:09.349759 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:09.349640 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42f7b319_86ae_4c55_adac_ab5c2a44ed68.slice/crio-42f08442ee66be8a4384ed9cc47fb1feefe23e8a1e8c064b533323057844879d WatchSource:0}: Error finding container 42f08442ee66be8a4384ed9cc47fb1feefe23e8a1e8c064b533323057844879d: Status 404 returned error can't find the container with id 42f08442ee66be8a4384ed9cc47fb1feefe23e8a1e8c064b533323057844879d Jul 26 17:49:09.350651 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:09.350572 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe73bb50_4e02_4441_9b09_da833761aac5.slice/crio-67ee34cd5d78eec36c9a0bb4ea3841873384b6039b68f88610704ad213f6cb09 WatchSource:0}: Error finding container 67ee34cd5d78eec36c9a0bb4ea3841873384b6039b68f88610704ad213f6cb09: Status 404 returned error can't find the container with id 67ee34cd5d78eec36c9a0bb4ea3841873384b6039b68f88610704ad213f6cb09 Jul 26 17:49:09.352032 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:09.352011 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod569a96e6_4c60_43e1_8419_59b3992e3d85.slice/crio-c8e6cd5425251207d32c2663baf1b9f1cf6ebfed021c3ffd3b5ff2baf151792d WatchSource:0}: Error finding container c8e6cd5425251207d32c2663baf1b9f1cf6ebfed021c3ffd3b5ff2baf151792d: Status 404 returned error can't find the container with id c8e6cd5425251207d32c2663baf1b9f1cf6ebfed021c3ffd3b5ff2baf151792d Jul 26 17:49:09.387830 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:09.387784 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-25 17:44:08 +0000 UTC" deadline="2028-02-10 22:50:52.956385109 +0000 UTC" Jul 26 17:49:09.387830 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:09.387826 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13541h1m43.568561349s" Jul 26 17:49:09.443346 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:09.443082 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-209.ec2.internal" event={"ID":"c79721d966a6282ec9f6039747948682","Type":"ContainerStarted","Data":"cb9ca49a653c3aedadcdd3448d85671fa2ffa639e8cdf7951ee3dee407c1b392"} Jul 26 17:49:09.444371 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:09.444344 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5cpxq" event={"ID":"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0","Type":"ContainerStarted","Data":"30571208678b544b167678d6cfa1b691f6041fc35e3699a77ea7693c795f2f5f"} Jul 26 17:49:09.445429 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:09.445401 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-msmbj" event={"ID":"23a42634-d5de-4d67-a10a-2f5d8872486c","Type":"ContainerStarted","Data":"78028ed9d6fcc7d75a28c528023eaaebe01e07fac1da0c0f5ac975a93da4c4f9"} Jul 26 17:49:09.446598 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:09.446577 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xnd6k" event={"ID":"be73bb50-4e02-4441-9b09-da833761aac5","Type":"ContainerStarted","Data":"67ee34cd5d78eec36c9a0bb4ea3841873384b6039b68f88610704ad213f6cb09"} Jul 26 17:49:09.447789 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:09.447765 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" event={"ID":"42f7b319-86ae-4c55-adac-ab5c2a44ed68","Type":"ContainerStarted","Data":"42f08442ee66be8a4384ed9cc47fb1feefe23e8a1e8c064b533323057844879d"} Jul 26 17:49:09.448889 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:09.448867 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bh5vf" event={"ID":"a5ef698f-5999-4902-9fe6-57b185586841","Type":"ContainerStarted","Data":"4a3c33806987cab72782b4e2c8e25345f1333b8e846e622cb6f949b0db7b06b6"} Jul 26 17:49:09.449946 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:09.449920 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2575c" event={"ID":"e0cf3a10-8a54-45ad-aafa-bb7175208585","Type":"ContainerStarted","Data":"4c8e813aa797792048978adb32465523d51dd8d772387ffd7fdb0d684077f2db"} Jul 26 17:49:09.450986 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:09.450966 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" event={"ID":"569a96e6-4c60-43e1-8419-59b3992e3d85","Type":"ContainerStarted","Data":"c8e6cd5425251207d32c2663baf1b9f1cf6ebfed021c3ffd3b5ff2baf151792d"} Jul 26 17:49:09.452145 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:09.452124 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-226rc" event={"ID":"0afb128f-7308-461d-b7d4-6fae2fcce66f","Type":"ContainerStarted","Data":"f457b3dad9e218d4f8d60601c18e200bba9d7e13ad5e87a1fa4198cb277d6760"} Jul 26 17:49:09.453189 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:09.453167 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4sdbx" event={"ID":"5c23228c-55bb-49e3-b76f-d22199c6a153","Type":"ContainerStarted","Data":"e444a40f4911b91092596c29425820e6de174d8f3c440fa2b881cebbcf725c55"} Jul 26 17:49:09.456461 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:09.456424 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-209.ec2.internal" podStartSLOduration=2.456413918 podStartE2EDuration="2.456413918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:09.456376001 +0000 UTC m=+3.409031689" watchObservedRunningTime="2026-07-26 17:49:09.456413918 +0000 UTC m=+3.409069606" Jul 26 17:49:09.972831 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:09.972739 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret\") pod \"global-pull-secret-syncer-fgwwx\" (UID: \"92dc3c1b-48ce-4b80-858c-5421f0bcc2f2\") " pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:09.972997 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:09.972897 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 26 17:49:09.972997 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:09.972964 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret podName:92dc3c1b-48ce-4b80-858c-5421f0bcc2f2 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:11.972944311 +0000 UTC m=+5.925599980 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret") pod "global-pull-secret-syncer-fgwwx" (UID: "92dc3c1b-48ce-4b80-858c-5421f0bcc2f2") : object "kube-system"/"original-pull-secret" not registered Jul 26 17:49:10.073691 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:10.073642 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs\") pod \"network-metrics-daemon-v5q7r\" (UID: \"b6d9352c-2170-418f-9836-89640e78d1e1\") " pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:10.073877 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:10.073805 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 17:49:10.073877 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:10.073867 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs podName:b6d9352c-2170-418f-9836-89640e78d1e1 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:12.073849559 +0000 UTC m=+6.026505232 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs") pod "network-metrics-daemon-v5q7r" (UID: "b6d9352c-2170-418f-9836-89640e78d1e1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 17:49:10.174928 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:10.174872 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5wz7\" (UniqueName: \"kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7\") pod \"network-check-target-2sjzw\" (UID: \"bab84b92-d4aa-422d-a364-195365d32cfb\") " pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:10.175166 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:10.175048 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 26 17:49:10.175166 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:10.175071 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 26 17:49:10.175166 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:10.175083 2559 projected.go:196] Error preparing data for projected volume kube-api-access-x5wz7 for pod openshift-network-diagnostics/network-check-target-2sjzw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 17:49:10.175166 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:10.175155 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7 podName:bab84b92-d4aa-422d-a364-195365d32cfb nodeName:}" failed. No retries permitted until 2026-07-26 17:49:12.175124922 +0000 UTC m=+6.127780610 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-x5wz7" (UniqueName: "kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7") pod "network-check-target-2sjzw" (UID: "bab84b92-d4aa-422d-a364-195365d32cfb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 17:49:10.388643 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:10.388551 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-25 17:44:08 +0000 UTC" deadline="2027-12-25 06:16:04.648381299 +0000 UTC" Jul 26 17:49:10.388643 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:10.388593 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12396h26m54.259792306s" Jul 26 17:49:10.438048 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:10.437139 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:10.438048 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:10.437273 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgwwx" podUID="92dc3c1b-48ce-4b80-858c-5421f0bcc2f2" Jul 26 17:49:10.438048 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:10.437650 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:10.438048 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:10.437770 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5q7r" podUID="b6d9352c-2170-418f-9836-89640e78d1e1" Jul 26 17:49:10.438048 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:10.437832 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:10.438048 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:10.437898 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2sjzw" podUID="bab84b92-d4aa-422d-a364-195365d32cfb" Jul 26 17:49:10.471596 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:10.471556 2559 generic.go:356] "Generic (PLEG): container finished" podID="9f64ddc5d2c114aec5edbd37b23d5007" containerID="355a30133b020f2ceb0c6f2db715079bd71c4edc4a9a623d4f8e6070e28aa479" exitCode=0 Jul 26 17:49:10.472518 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:10.472490 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-209.ec2.internal" event={"ID":"9f64ddc5d2c114aec5edbd37b23d5007","Type":"ContainerDied","Data":"355a30133b020f2ceb0c6f2db715079bd71c4edc4a9a623d4f8e6070e28aa479"} Jul 26 17:49:11.491341 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:11.490549 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-209.ec2.internal" event={"ID":"9f64ddc5d2c114aec5edbd37b23d5007","Type":"ContainerStarted","Data":"eceecab98b8b76a63c545dfe9d0415f9f69b53bb87bb061159b1c4602e68d969"} Jul 26 17:49:11.990974 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:11.990731 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret\") pod \"global-pull-secret-syncer-fgwwx\" (UID: \"92dc3c1b-48ce-4b80-858c-5421f0bcc2f2\") " pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:11.991178 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:11.990882 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 26 17:49:11.991178 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:11.991059 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret podName:92dc3c1b-48ce-4b80-858c-5421f0bcc2f2 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:15.991029312 +0000 UTC m=+9.943684984 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret") pod "global-pull-secret-syncer-fgwwx" (UID: "92dc3c1b-48ce-4b80-858c-5421f0bcc2f2") : object "kube-system"/"original-pull-secret" not registered Jul 26 17:49:12.092904 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:12.092312 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs\") pod \"network-metrics-daemon-v5q7r\" (UID: \"b6d9352c-2170-418f-9836-89640e78d1e1\") " pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:12.092904 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:12.092452 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 17:49:12.092904 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:12.092525 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs podName:b6d9352c-2170-418f-9836-89640e78d1e1 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:16.092505691 +0000 UTC m=+10.045161360 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs") pod "network-metrics-daemon-v5q7r" (UID: "b6d9352c-2170-418f-9836-89640e78d1e1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 17:49:12.193299 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:12.193259 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5wz7\" (UniqueName: \"kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7\") pod \"network-check-target-2sjzw\" (UID: \"bab84b92-d4aa-422d-a364-195365d32cfb\") " pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:12.193475 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:12.193457 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 26 17:49:12.193538 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:12.193481 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 26 17:49:12.193538 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:12.193493 2559 projected.go:196] Error preparing data for projected volume kube-api-access-x5wz7 for pod openshift-network-diagnostics/network-check-target-2sjzw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 17:49:12.193636 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:12.193556 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7 podName:bab84b92-d4aa-422d-a364-195365d32cfb nodeName:}" failed. No retries permitted until 2026-07-26 17:49:16.193542003 +0000 UTC m=+10.146197669 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-x5wz7" (UniqueName: "kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7") pod "network-check-target-2sjzw" (UID: "bab84b92-d4aa-422d-a364-195365d32cfb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 17:49:12.434546 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:12.433832 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:12.434546 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:12.433832 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:12.434546 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:12.434053 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5q7r" podUID="b6d9352c-2170-418f-9836-89640e78d1e1" Jul 26 17:49:12.434546 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:12.434146 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgwwx" podUID="92dc3c1b-48ce-4b80-858c-5421f0bcc2f2" Jul 26 17:49:12.434546 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:12.434224 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:12.434546 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:12.434297 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2sjzw" podUID="bab84b92-d4aa-422d-a364-195365d32cfb" Jul 26 17:49:14.433164 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:14.433132 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:14.433612 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:14.433205 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:14.433612 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:14.433133 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:14.433612 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:14.433285 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5q7r" podUID="b6d9352c-2170-418f-9836-89640e78d1e1" Jul 26 17:49:14.433612 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:14.433415 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgwwx" podUID="92dc3c1b-48ce-4b80-858c-5421f0bcc2f2" Jul 26 17:49:14.433612 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:14.433544 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2sjzw" podUID="bab84b92-d4aa-422d-a364-195365d32cfb" Jul 26 17:49:16.029064 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:16.029025 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret\") pod \"global-pull-secret-syncer-fgwwx\" (UID: \"92dc3c1b-48ce-4b80-858c-5421f0bcc2f2\") " pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:16.029580 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:16.029249 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 26 17:49:16.029580 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:16.029332 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret podName:92dc3c1b-48ce-4b80-858c-5421f0bcc2f2 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:24.029308412 +0000 UTC m=+17.981964080 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret") pod "global-pull-secret-syncer-fgwwx" (UID: "92dc3c1b-48ce-4b80-858c-5421f0bcc2f2") : object "kube-system"/"original-pull-secret" not registered Jul 26 17:49:16.130073 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:16.130014 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs\") pod \"network-metrics-daemon-v5q7r\" (UID: \"b6d9352c-2170-418f-9836-89640e78d1e1\") " pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:16.130251 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:16.130156 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 17:49:16.130251 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:16.130250 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs podName:b6d9352c-2170-418f-9836-89640e78d1e1 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:24.130228709 +0000 UTC m=+18.082884398 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs") pod "network-metrics-daemon-v5q7r" (UID: "b6d9352c-2170-418f-9836-89640e78d1e1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 17:49:16.231373 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:16.231322 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5wz7\" (UniqueName: \"kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7\") pod \"network-check-target-2sjzw\" (UID: \"bab84b92-d4aa-422d-a364-195365d32cfb\") " pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:16.231569 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:16.231511 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 26 17:49:16.231569 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:16.231533 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 26 17:49:16.231569 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:16.231548 2559 projected.go:196] Error preparing data for projected volume kube-api-access-x5wz7 for pod openshift-network-diagnostics/network-check-target-2sjzw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 17:49:16.231756 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:16.231605 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7 podName:bab84b92-d4aa-422d-a364-195365d32cfb nodeName:}" failed. No retries permitted until 2026-07-26 17:49:24.231587137 +0000 UTC m=+18.184242830 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-x5wz7" (UniqueName: "kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7") pod "network-check-target-2sjzw" (UID: "bab84b92-d4aa-422d-a364-195365d32cfb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 17:49:16.437302 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:16.434100 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:16.437302 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:16.434276 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:16.437302 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:16.434415 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5q7r" podUID="b6d9352c-2170-418f-9836-89640e78d1e1" Jul 26 17:49:16.437302 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:16.434889 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgwwx" podUID="92dc3c1b-48ce-4b80-858c-5421f0bcc2f2" Jul 26 17:49:16.437302 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:16.434950 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:16.437302 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:16.435067 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2sjzw" podUID="bab84b92-d4aa-422d-a364-195365d32cfb" Jul 26 17:49:18.433017 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:18.432923 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:18.433017 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:18.432943 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:18.433589 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:18.432923 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:18.433589 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:18.433071 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgwwx" podUID="92dc3c1b-48ce-4b80-858c-5421f0bcc2f2" Jul 26 17:49:18.433589 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:18.433191 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2sjzw" podUID="bab84b92-d4aa-422d-a364-195365d32cfb" Jul 26 17:49:18.433589 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:18.433292 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5q7r" podUID="b6d9352c-2170-418f-9836-89640e78d1e1" Jul 26 17:49:20.433405 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:20.433372 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:20.433873 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:20.433379 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:20.433873 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:20.433499 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2sjzw" podUID="bab84b92-d4aa-422d-a364-195365d32cfb" Jul 26 17:49:20.433873 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:20.433553 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgwwx" podUID="92dc3c1b-48ce-4b80-858c-5421f0bcc2f2" Jul 26 17:49:20.433873 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:20.433387 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:20.433873 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:20.433654 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5q7r" podUID="b6d9352c-2170-418f-9836-89640e78d1e1" Jul 26 17:49:22.432848 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:22.432811 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:22.433315 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:22.432809 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:22.433315 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:22.432947 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:22.433315 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:22.432942 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2sjzw" podUID="bab84b92-d4aa-422d-a364-195365d32cfb" Jul 26 17:49:22.433315 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:22.433074 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5q7r" podUID="b6d9352c-2170-418f-9836-89640e78d1e1" Jul 26 17:49:22.433315 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:22.433169 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgwwx" podUID="92dc3c1b-48ce-4b80-858c-5421f0bcc2f2" Jul 26 17:49:24.092045 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:24.092011 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret\") pod \"global-pull-secret-syncer-fgwwx\" (UID: \"92dc3c1b-48ce-4b80-858c-5421f0bcc2f2\") " pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:24.092530 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:24.092137 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 26 17:49:24.092530 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:24.092202 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret podName:92dc3c1b-48ce-4b80-858c-5421f0bcc2f2 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:40.092181681 +0000 UTC m=+34.044837370 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret") pod "global-pull-secret-syncer-fgwwx" (UID: "92dc3c1b-48ce-4b80-858c-5421f0bcc2f2") : object "kube-system"/"original-pull-secret" not registered Jul 26 17:49:24.192648 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:24.192616 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs\") pod \"network-metrics-daemon-v5q7r\" (UID: \"b6d9352c-2170-418f-9836-89640e78d1e1\") " pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:24.192848 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:24.192745 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 17:49:24.192848 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:24.192813 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs podName:b6d9352c-2170-418f-9836-89640e78d1e1 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:40.192793384 +0000 UTC m=+34.145449082 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs") pod "network-metrics-daemon-v5q7r" (UID: "b6d9352c-2170-418f-9836-89640e78d1e1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 17:49:24.293311 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:24.293268 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5wz7\" (UniqueName: \"kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7\") pod \"network-check-target-2sjzw\" (UID: \"bab84b92-d4aa-422d-a364-195365d32cfb\") " pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:24.293496 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:24.293466 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 26 17:49:24.293496 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:24.293496 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 26 17:49:24.293600 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:24.293509 2559 projected.go:196] Error preparing data for projected volume kube-api-access-x5wz7 for pod openshift-network-diagnostics/network-check-target-2sjzw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 17:49:24.293600 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:24.293576 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7 podName:bab84b92-d4aa-422d-a364-195365d32cfb nodeName:}" failed. No retries permitted until 2026-07-26 17:49:40.293555405 +0000 UTC m=+34.246211078 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-x5wz7" (UniqueName: "kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7") pod "network-check-target-2sjzw" (UID: "bab84b92-d4aa-422d-a364-195365d32cfb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 17:49:24.433221 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:24.433135 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:24.433372 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:24.433135 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:24.433372 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:24.433260 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgwwx" podUID="92dc3c1b-48ce-4b80-858c-5421f0bcc2f2" Jul 26 17:49:24.433372 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:24.433143 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:24.433372 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:24.433338 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2sjzw" podUID="bab84b92-d4aa-422d-a364-195365d32cfb" Jul 26 17:49:24.433580 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:24.433455 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5q7r" podUID="b6d9352c-2170-418f-9836-89640e78d1e1" Jul 26 17:49:26.433439 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:26.433408 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:26.433954 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:26.433522 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5q7r" podUID="b6d9352c-2170-418f-9836-89640e78d1e1" Jul 26 17:49:26.433954 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:26.433726 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:26.433954 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:26.433815 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:26.433954 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:26.433921 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2sjzw" podUID="bab84b92-d4aa-422d-a364-195365d32cfb" Jul 26 17:49:26.434169 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:26.434003 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgwwx" podUID="92dc3c1b-48ce-4b80-858c-5421f0bcc2f2" Jul 26 17:49:28.433598 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:28.433284 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:28.434413 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:28.433324 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:28.434413 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:28.433717 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgwwx" podUID="92dc3c1b-48ce-4b80-858c-5421f0bcc2f2" Jul 26 17:49:28.434413 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:28.433284 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:28.434413 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:28.433791 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5q7r" podUID="b6d9352c-2170-418f-9836-89640e78d1e1" Jul 26 17:49:28.434413 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:28.433833 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2sjzw" podUID="bab84b92-d4aa-422d-a364-195365d32cfb" Jul 26 17:49:28.526983 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:28.526951 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-226rc" event={"ID":"0afb128f-7308-461d-b7d4-6fae2fcce66f","Type":"ContainerStarted","Data":"a33087210867033bf12d021e6aba75f9955972e90f0b092755278be40bfac597"} Jul 26 17:49:28.528481 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:28.528457 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4sdbx" event={"ID":"5c23228c-55bb-49e3-b76f-d22199c6a153","Type":"ContainerStarted","Data":"087475b0d9ccc962514ca62bce66aa4381a456f5e0f30ab83539523c75670166"} Jul 26 17:49:28.529989 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:28.529963 2559 generic.go:356] "Generic (PLEG): container finished" podID="0c0e8dd9-a868-44f9-bec5-aa0aaae878d0" containerID="52323767eb42d3cba2817fa5e14e10ee6479dd30f0bfacc56e8817430c4551cf" exitCode=0 Jul 26 17:49:28.530107 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:28.530033 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5cpxq" event={"ID":"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0","Type":"ContainerDied","Data":"52323767eb42d3cba2817fa5e14e10ee6479dd30f0bfacc56e8817430c4551cf"} Jul 26 17:49:28.531512 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:28.531488 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xnd6k" event={"ID":"be73bb50-4e02-4441-9b09-da833761aac5","Type":"ContainerStarted","Data":"d8e56ef291e0b6a992965afd01cafdc459ed25ff5bcd8645183d36be10661a4a"} Jul 26 17:49:28.534112 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:28.534086 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" event={"ID":"42f7b319-86ae-4c55-adac-ab5c2a44ed68","Type":"ContainerStarted","Data":"a287e2df039058b3c10c1999da89717849e6996e32f9c1574f6cb19d39c51072"} Jul 26 17:49:28.534226 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:28.534164 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" event={"ID":"42f7b319-86ae-4c55-adac-ab5c2a44ed68","Type":"ContainerStarted","Data":"3a3c14a5620e4fdbb8ed8ecc6b08dbc54a27061a6ec687d532650d2b53c8c071"} Jul 26 17:49:28.534293 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:28.534229 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" event={"ID":"42f7b319-86ae-4c55-adac-ab5c2a44ed68","Type":"ContainerStarted","Data":"787f041bb08c64d6639559b26ce0525864d885238ed9b0635db1e4e382580d23"} Jul 26 17:49:28.534293 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:28.534280 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" event={"ID":"42f7b319-86ae-4c55-adac-ab5c2a44ed68","Type":"ContainerStarted","Data":"ed869d84bf1d4818a1d79685db83ffc85698c930dc67796ab5249f747b079bd2"} Jul 26 17:49:28.535455 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:28.535431 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bh5vf" event={"ID":"a5ef698f-5999-4902-9fe6-57b185586841","Type":"ContainerStarted","Data":"c2ad2158add5810f5d4f8ba0849e26cb77afd5b52cb2c5c77e5b75ca5e411721"} Jul 26 17:49:28.536829 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:28.536795 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2575c" event={"ID":"e0cf3a10-8a54-45ad-aafa-bb7175208585","Type":"ContainerStarted","Data":"44ec6f226f82ee659a8c0f5d856c4614fc633569f1542be0fd36dad3fd148efe"} Jul 26 17:49:28.538252 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:28.538228 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" event={"ID":"569a96e6-4c60-43e1-8419-59b3992e3d85","Type":"ContainerStarted","Data":"fbde0cee25638b2c13a4249a4c5465db856431c91c6ddd519dc8ccf9cd3fc907"} Jul 26 17:49:28.542196 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:28.542157 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-226rc" podStartSLOduration=3.840095307 podStartE2EDuration="22.54214712s" totalImagesPullingTime="18.702051813s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:28.542125802 +0000 UTC m=+22.494781489" watchObservedRunningTime="2026-07-26 17:49:28.54214712 +0000 UTC m=+22.494802808" Jul 26 17:49:28.542565 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:28.542544 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-209.ec2.internal" podStartSLOduration=21.54253698 podStartE2EDuration="21.54253698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:11.507097669 +0000 UTC m=+5.459753360" watchObservedRunningTime="2026-07-26 17:49:28.54253698 +0000 UTC m=+22.495192667" Jul 26 17:49:28.555469 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:28.555424 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2575c" podStartSLOduration=3.864694538 podStartE2EDuration="22.555412638s" totalImagesPullingTime="18.6907181s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:28.555104914 +0000 UTC m=+22.507760601" watchObservedRunningTime="2026-07-26 17:49:28.555412638 +0000 UTC m=+22.508068325" Jul 26 17:49:28.590148 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:28.590102 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4sdbx" podStartSLOduration=12.077049134 podStartE2EDuration="22.590085828s" totalImagesPullingTime="10.513036694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:28.590074355 +0000 UTC m=+22.542730042" watchObservedRunningTime="2026-07-26 17:49:28.590085828 +0000 UTC m=+22.542741510" Jul 26 17:49:28.604885 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:28.604837 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xnd6k" podStartSLOduration=3.812840779 podStartE2EDuration="22.604820048s" totalImagesPullingTime="18.791979269s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:28.604539926 +0000 UTC m=+22.557195614" watchObservedRunningTime="2026-07-26 17:49:28.604820048 +0000 UTC m=+22.557475735" Jul 26 17:49:29.538035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:29.538011 2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 26 17:49:29.545615 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:29.545578 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-msmbj" event={"ID":"23a42634-d5de-4d67-a10a-2f5d8872486c","Type":"ContainerStarted","Data":"b95aa6056eec890d9ba5dc9a5fbf579340e2843b3e094c1d9a99afa68ff7dc58"} Jul 26 17:49:29.549982 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:29.549944 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" event={"ID":"42f7b319-86ae-4c55-adac-ab5c2a44ed68","Type":"ContainerStarted","Data":"f4e001522fd959abf8ad0d9677815cf9530111406422432213494f8764052848"} Jul 26 17:49:29.550092 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:29.550016 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" event={"ID":"42f7b319-86ae-4c55-adac-ab5c2a44ed68","Type":"ContainerStarted","Data":"e1b10714ca2846934d01e25a29cb492f04cd6ea1a3c29653eba289670faf1129"} Jul 26 17:49:29.551937 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:29.551909 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" event={"ID":"569a96e6-4c60-43e1-8419-59b3992e3d85","Type":"ContainerStarted","Data":"41a3ff59b724a8802fc86614752759e75ff0386931bc5ea18c92426075e72819"} Jul 26 17:49:29.560564 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:29.560513 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bh5vf" podStartSLOduration=4.854539054 podStartE2EDuration="23.560492157s" totalImagesPullingTime="18.705953103s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:28.620224131 +0000 UTC m=+22.572879818" watchObservedRunningTime="2026-07-26 17:49:29.560492157 +0000 UTC m=+23.513147907" Jul 26 17:49:30.422709 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:30.422558 2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-26T17:49:29.53802907Z","UUID":"a1241b0e-c186-4e7d-9024-2be25780fcad","Handler":null,"Name":"","Endpoint":""} Jul 26 17:49:30.424485 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:30.424464 2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 26 17:49:30.424633 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:30.424506 2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 26 17:49:30.432802 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:30.432771 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:30.432994 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:30.432776 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:30.432994 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:30.432898 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5q7r" podUID="b6d9352c-2170-418f-9836-89640e78d1e1" Jul 26 17:49:30.433117 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:30.433013 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2sjzw" podUID="bab84b92-d4aa-422d-a364-195365d32cfb" Jul 26 17:49:30.433117 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:30.433078 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:30.433222 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:30.433154 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgwwx" podUID="92dc3c1b-48ce-4b80-858c-5421f0bcc2f2" Jul 26 17:49:31.084513 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:31.084089 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4sdbx" Jul 26 17:49:31.085271 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:31.085248 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4sdbx" Jul 26 17:49:31.103199 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:31.103150 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-msmbj" podStartSLOduration=6.407728938 podStartE2EDuration="25.10313375s" totalImagesPullingTime="18.695404812s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:29.560165694 +0000 UTC m=+23.512821387" watchObservedRunningTime="2026-07-26 17:49:31.10313375 +0000 UTC m=+25.055789440" Jul 26 17:49:31.558983 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:31.558890 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" event={"ID":"569a96e6-4c60-43e1-8419-59b3992e3d85","Type":"ContainerStarted","Data":"b11f1c4782c51e5f91246802a1d32748188b898244796193f9ca67e6e62b5828"} Jul 26 17:49:31.562545 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:31.562516 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" event={"ID":"42f7b319-86ae-4c55-adac-ab5c2a44ed68","Type":"ContainerStarted","Data":"7880f41df7a850eb0e337a84ca7ce872099e18f4ac9a8a7f6172c54b7bffabbf"} Jul 26 17:49:31.579971 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:31.579898 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74r2h" podStartSLOduration=4.654868238 podStartE2EDuration="25.57988232s" totalImagesPullingTime="20.925014082s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:31.579699997 +0000 UTC m=+25.532355705" watchObservedRunningTime="2026-07-26 17:49:31.57988232 +0000 UTC m=+25.532538007" Jul 26 17:49:31.582818 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:31.582793 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4sdbx" Jul 26 17:49:31.583400 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:31.583378 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4sdbx" Jul 26 17:49:32.433733 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:32.433696 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:32.434322 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:32.433696 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:32.434322 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:32.433827 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2sjzw" podUID="bab84b92-d4aa-422d-a364-195365d32cfb" Jul 26 17:49:32.434322 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:32.433856 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:32.434322 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:32.433939 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5q7r" podUID="b6d9352c-2170-418f-9836-89640e78d1e1" Jul 26 17:49:32.434322 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:32.434014 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgwwx" podUID="92dc3c1b-48ce-4b80-858c-5421f0bcc2f2" Jul 26 17:49:33.568542 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:33.568352 2559 generic.go:356] "Generic (PLEG): container finished" podID="0c0e8dd9-a868-44f9-bec5-aa0aaae878d0" containerID="e5b57828175b22264f7f19f5d311d05dc8b8ddffb8b88d9ee06f9ab82a4cae99" exitCode=0 Jul 26 17:49:33.569309 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:33.568442 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5cpxq" event={"ID":"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0","Type":"ContainerDied","Data":"e5b57828175b22264f7f19f5d311d05dc8b8ddffb8b88d9ee06f9ab82a4cae99"} Jul 26 17:49:33.572330 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:33.572307 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" event={"ID":"42f7b319-86ae-4c55-adac-ab5c2a44ed68","Type":"ContainerStarted","Data":"e43a1ce691cd68d0c346749df87ecced1e41256bb89478e9559ce9611582a944"} Jul 26 17:49:33.620905 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:33.620846 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" podStartSLOduration=8.874454546 podStartE2EDuration="27.620828842s" totalImagesPullingTime="18.746374296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:33.620298349 +0000 UTC m=+27.572954048" watchObservedRunningTime="2026-07-26 17:49:33.620828842 +0000 UTC m=+27.573484529" Jul 26 17:49:34.032421 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:34.032386 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:34.436142 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:34.435266 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:34.436142 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:34.435402 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgwwx" podUID="92dc3c1b-48ce-4b80-858c-5421f0bcc2f2" Jul 26 17:49:34.436142 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:34.435821 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:34.436142 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:34.435928 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5q7r" podUID="b6d9352c-2170-418f-9836-89640e78d1e1" Jul 26 17:49:34.436142 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:34.435994 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:34.436142 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:34.436070 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2sjzw" podUID="bab84b92-d4aa-422d-a364-195365d32cfb" Jul 26 17:49:34.575052 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:34.574985 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:34.575469 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:34.575121 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:34.590480 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:34.590455 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:34.590661 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:34.590624 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:49:35.365325 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:35.365283 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2sjzw"] Jul 26 17:49:35.365501 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:35.365435 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:35.365565 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:35.365531 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2sjzw" podUID="bab84b92-d4aa-422d-a364-195365d32cfb" Jul 26 17:49:35.368932 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:35.368899 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v5q7r"] Jul 26 17:49:35.369048 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:35.369021 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:35.369152 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:35.369131 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5q7r" podUID="b6d9352c-2170-418f-9836-89640e78d1e1" Jul 26 17:49:35.377400 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:35.377343 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fgwwx"] Jul 26 17:49:35.377482 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:35.377426 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:35.377519 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:35.377494 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgwwx" podUID="92dc3c1b-48ce-4b80-858c-5421f0bcc2f2" Jul 26 17:49:35.578929 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:35.578891 2559 generic.go:356] "Generic (PLEG): container finished" podID="0c0e8dd9-a868-44f9-bec5-aa0aaae878d0" containerID="ec93c467820dad1e819098592c765e121fc11f257c9dc0cdc8a9edee072076f7" exitCode=0 Jul 26 17:49:35.579302 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:35.578982 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5cpxq" event={"ID":"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0","Type":"ContainerDied","Data":"ec93c467820dad1e819098592c765e121fc11f257c9dc0cdc8a9edee072076f7"} Jul 26 17:49:36.433241 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:36.433215 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:36.433405 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:36.433298 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgwwx" podUID="92dc3c1b-48ce-4b80-858c-5421f0bcc2f2" Jul 26 17:49:37.432995 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:37.432767 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:37.433341 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:37.432797 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:37.433341 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:37.433012 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2sjzw" podUID="bab84b92-d4aa-422d-a364-195365d32cfb" Jul 26 17:49:37.433341 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:37.433094 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5q7r" podUID="b6d9352c-2170-418f-9836-89640e78d1e1" Jul 26 17:49:37.586690 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:37.586635 2559 generic.go:356] "Generic (PLEG): container finished" podID="0c0e8dd9-a868-44f9-bec5-aa0aaae878d0" containerID="e911789136c193ac03831d1dbfe02dc708cf0a5da2f3216d22922626aa10fb28" exitCode=0 Jul 26 17:49:37.586851 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:37.586711 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5cpxq" event={"ID":"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0","Type":"ContainerDied","Data":"e911789136c193ac03831d1dbfe02dc708cf0a5da2f3216d22922626aa10fb28"} Jul 26 17:49:38.433498 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:38.433467 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:38.433963 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:38.433588 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgwwx" podUID="92dc3c1b-48ce-4b80-858c-5421f0bcc2f2" Jul 26 17:49:39.433297 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:39.433267 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:39.433458 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:39.433310 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:39.433458 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:39.433377 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2sjzw" podUID="bab84b92-d4aa-422d-a364-195365d32cfb" Jul 26 17:49:39.433558 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:39.433462 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5q7r" podUID="b6d9352c-2170-418f-9836-89640e78d1e1" Jul 26 17:49:39.656603 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:39.656498 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:39.755836 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:39.755784 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:39.850199 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:39.850153 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:40.112562 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.112465 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret\") pod \"global-pull-secret-syncer-fgwwx\" (UID: \"92dc3c1b-48ce-4b80-858c-5421f0bcc2f2\") " pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:40.112757 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:40.112630 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 26 17:49:40.112757 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:40.112725 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret podName:92dc3c1b-48ce-4b80-858c-5421f0bcc2f2 nodeName:}" failed. No retries permitted until 2026-07-26 17:50:12.112705882 +0000 UTC m=+66.065361554 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret") pod "global-pull-secret-syncer-fgwwx" (UID: "92dc3c1b-48ce-4b80-858c-5421f0bcc2f2") : object "kube-system"/"original-pull-secret" not registered Jul 26 17:49:40.213280 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.213244 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs\") pod \"network-metrics-daemon-v5q7r\" (UID: \"b6d9352c-2170-418f-9836-89640e78d1e1\") " pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:40.213462 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:40.213425 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 17:49:40.213551 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:40.213506 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs podName:b6d9352c-2170-418f-9836-89640e78d1e1 nodeName:}" failed. No retries permitted until 2026-07-26 17:50:12.213483625 +0000 UTC m=+66.166139345 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs") pod "network-metrics-daemon-v5q7r" (UID: "b6d9352c-2170-418f-9836-89640e78d1e1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 26 17:49:40.314531 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.314483 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5wz7\" (UniqueName: \"kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7\") pod \"network-check-target-2sjzw\" (UID: \"bab84b92-d4aa-422d-a364-195365d32cfb\") " pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:40.314708 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:40.314660 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 26 17:49:40.314708 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:40.314694 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 26 17:49:40.314708 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:40.314708 2559 projected.go:196] Error preparing data for projected volume kube-api-access-x5wz7 for pod openshift-network-diagnostics/network-check-target-2sjzw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 17:49:40.314860 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:40.314770 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7 podName:bab84b92-d4aa-422d-a364-195365d32cfb nodeName:}" failed. No retries permitted until 2026-07-26 17:50:12.314751143 +0000 UTC m=+66.267406811 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-x5wz7" (UniqueName: "kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7") pod "network-check-target-2sjzw" (UID: "bab84b92-d4aa-422d-a364-195365d32cfb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 26 17:49:40.433326 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.433244 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:40.433485 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:40.433372 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fgwwx" podUID="92dc3c1b-48ce-4b80-858c-5421f0bcc2f2" Jul 26 17:49:40.850829 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.850750 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-209.ec2.internal" event="NodeReady" Jul 26 17:49:40.851372 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.850943 2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 26 17:49:40.886022 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.885987 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-68cdbc4cd7-qj2k4"] Jul 26 17:49:40.891467 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.891442 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:40.894005 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.893923 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 26 17:49:40.894005 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.893992 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 26 17:49:40.894005 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.894006 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2hr9p\"" Jul 26 17:49:40.894349 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.894227 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 26 17:49:40.895729 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.895708 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5q7q7"] Jul 26 17:49:40.899405 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.899386 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5q7q7" Jul 26 17:49:40.900851 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.899879 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bgs6z"] Jul 26 17:49:40.902100 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.902046 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 26 17:49:40.902100 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.902079 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 26 17:49:40.902495 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.902479 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jb7dx\"" Jul 26 17:49:40.902571 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.902487 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 26 17:49:40.904118 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.904090 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68cdbc4cd7-qj2k4"] Jul 26 17:49:40.904205 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.904191 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bgs6z" Jul 26 17:49:40.906971 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.906635 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-btq2h\"" Jul 26 17:49:40.906971 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.906717 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 26 17:49:40.906971 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.906773 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 26 17:49:40.907359 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.907342 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 26 17:49:40.912286 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.912267 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bgs6z"] Jul 26 17:49:40.924357 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:40.924334 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5q7q7"] Jul 26 17:49:41.020072 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.020032 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-bound-sa-token\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.020072 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.020083 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0b76c942-35c5-49e9-a469-db6ccacf23ff-ca-trust-extracted\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.020336 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.020120 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz6js\" (UniqueName: \"kubernetes.io/projected/1f1f2371-6ad2-4065-929a-d4e87827fa52-kube-api-access-fz6js\") pod \"dns-default-5q7q7\" (UID: \"1f1f2371-6ad2-4065-929a-d4e87827fa52\") " pod="openshift-dns/dns-default-5q7q7" Jul 26 17:49:41.020336 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.020233 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0b76c942-35c5-49e9-a469-db6ccacf23ff-installation-pull-secrets\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.020336 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.020280 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0b76c942-35c5-49e9-a469-db6ccacf23ff-image-registry-private-configuration\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.020481 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.020335 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0b76c942-35c5-49e9-a469-db6ccacf23ff-trusted-ca\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.020481 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.020379 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f1f2371-6ad2-4065-929a-d4e87827fa52-tmp-dir\") pod \"dns-default-5q7q7\" (UID: \"1f1f2371-6ad2-4065-929a-d4e87827fa52\") " pod="openshift-dns/dns-default-5q7q7" Jul 26 17:49:41.020481 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.020428 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c3ba442d-4651-44c3-b29d-14d536188970-cert\") pod \"ingress-canary-bgs6z\" (UID: \"c3ba442d-4651-44c3-b29d-14d536188970\") " pod="openshift-ingress-canary/ingress-canary-bgs6z" Jul 26 17:49:41.020481 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.020451 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zfkl\" (UniqueName: \"kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-kube-api-access-2zfkl\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.020649 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.020528 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-tls\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.020649 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.020561 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-certificates\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.020649 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.020597 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q288c\" (UniqueName: \"kubernetes.io/projected/c3ba442d-4651-44c3-b29d-14d536188970-kube-api-access-q288c\") pod \"ingress-canary-bgs6z\" (UID: \"c3ba442d-4651-44c3-b29d-14d536188970\") " pod="openshift-ingress-canary/ingress-canary-bgs6z" Jul 26 17:49:41.020649 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.020626 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1f1f2371-6ad2-4065-929a-d4e87827fa52-config-volume\") pod \"dns-default-5q7q7\" (UID: \"1f1f2371-6ad2-4065-929a-d4e87827fa52\") " pod="openshift-dns/dns-default-5q7q7" Jul 26 17:49:41.020855 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.020696 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f1f2371-6ad2-4065-929a-d4e87827fa52-metrics-tls\") pod \"dns-default-5q7q7\" (UID: \"1f1f2371-6ad2-4065-929a-d4e87827fa52\") " pod="openshift-dns/dns-default-5q7q7" Jul 26 17:49:41.121079 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.120995 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1f1f2371-6ad2-4065-929a-d4e87827fa52-config-volume\") pod \"dns-default-5q7q7\" (UID: \"1f1f2371-6ad2-4065-929a-d4e87827fa52\") " pod="openshift-dns/dns-default-5q7q7" Jul 26 17:49:41.121079 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.121039 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f1f2371-6ad2-4065-929a-d4e87827fa52-metrics-tls\") pod \"dns-default-5q7q7\" (UID: \"1f1f2371-6ad2-4065-929a-d4e87827fa52\") " pod="openshift-dns/dns-default-5q7q7" Jul 26 17:49:41.121079 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.121070 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-bound-sa-token\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.121346 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.121096 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0b76c942-35c5-49e9-a469-db6ccacf23ff-ca-trust-extracted\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.121346 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.121126 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz6js\" (UniqueName: \"kubernetes.io/projected/1f1f2371-6ad2-4065-929a-d4e87827fa52-kube-api-access-fz6js\") pod \"dns-default-5q7q7\" (UID: \"1f1f2371-6ad2-4065-929a-d4e87827fa52\") " pod="openshift-dns/dns-default-5q7q7" Jul 26 17:49:41.121346 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:41.121166 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 26 17:49:41.121346 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:41.121236 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f1f2371-6ad2-4065-929a-d4e87827fa52-metrics-tls podName:1f1f2371-6ad2-4065-929a-d4e87827fa52 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:41.621217897 +0000 UTC m=+35.573873581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1f1f2371-6ad2-4065-929a-d4e87827fa52-metrics-tls") pod "dns-default-5q7q7" (UID: "1f1f2371-6ad2-4065-929a-d4e87827fa52") : secret "dns-default-metrics-tls" not found Jul 26 17:49:41.121346 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.121169 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0b76c942-35c5-49e9-a469-db6ccacf23ff-installation-pull-secrets\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.121346 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.121325 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0b76c942-35c5-49e9-a469-db6ccacf23ff-image-registry-private-configuration\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.121634 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.121361 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0b76c942-35c5-49e9-a469-db6ccacf23ff-trusted-ca\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.121634 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.121387 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f1f2371-6ad2-4065-929a-d4e87827fa52-tmp-dir\") pod \"dns-default-5q7q7\" (UID: \"1f1f2371-6ad2-4065-929a-d4e87827fa52\") " pod="openshift-dns/dns-default-5q7q7" Jul 26 17:49:41.121634 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.121426 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c3ba442d-4651-44c3-b29d-14d536188970-cert\") pod \"ingress-canary-bgs6z\" (UID: \"c3ba442d-4651-44c3-b29d-14d536188970\") " pod="openshift-ingress-canary/ingress-canary-bgs6z" Jul 26 17:49:41.121634 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.121450 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zfkl\" (UniqueName: \"kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-kube-api-access-2zfkl\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.121634 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.121493 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-tls\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.121634 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.121517 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-certificates\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.121634 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.121524 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0b76c942-35c5-49e9-a469-db6ccacf23ff-ca-trust-extracted\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.121634 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.121555 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q288c\" (UniqueName: \"kubernetes.io/projected/c3ba442d-4651-44c3-b29d-14d536188970-kube-api-access-q288c\") pod \"ingress-canary-bgs6z\" (UID: \"c3ba442d-4651-44c3-b29d-14d536188970\") " pod="openshift-ingress-canary/ingress-canary-bgs6z" Jul 26 17:49:41.122012 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.121804 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f1f2371-6ad2-4065-929a-d4e87827fa52-tmp-dir\") pod \"dns-default-5q7q7\" (UID: \"1f1f2371-6ad2-4065-929a-d4e87827fa52\") " pod="openshift-dns/dns-default-5q7q7" Jul 26 17:49:41.122012 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.121825 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1f1f2371-6ad2-4065-929a-d4e87827fa52-config-volume\") pod \"dns-default-5q7q7\" (UID: \"1f1f2371-6ad2-4065-929a-d4e87827fa52\") " pod="openshift-dns/dns-default-5q7q7" Jul 26 17:49:41.122012 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:41.121904 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 26 17:49:41.122012 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:41.121919 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68cdbc4cd7-qj2k4: secret "image-registry-tls" not found Jul 26 17:49:41.122012 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:41.121978 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-tls podName:0b76c942-35c5-49e9-a469-db6ccacf23ff nodeName:}" failed. No retries permitted until 2026-07-26 17:49:41.621960106 +0000 UTC m=+35.574615777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-tls") pod "image-registry-68cdbc4cd7-qj2k4" (UID: "0b76c942-35c5-49e9-a469-db6ccacf23ff") : secret "image-registry-tls" not found Jul 26 17:49:41.122685 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.122626 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-certificates\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.123034 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.122987 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0b76c942-35c5-49e9-a469-db6ccacf23ff-trusted-ca\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.126169 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.126126 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0b76c942-35c5-49e9-a469-db6ccacf23ff-image-registry-private-configuration\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.126275 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.126197 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0b76c942-35c5-49e9-a469-db6ccacf23ff-installation-pull-secrets\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.126275 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.126206 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c3ba442d-4651-44c3-b29d-14d536188970-cert\") pod \"ingress-canary-bgs6z\" (UID: \"c3ba442d-4651-44c3-b29d-14d536188970\") " pod="openshift-ingress-canary/ingress-canary-bgs6z" Jul 26 17:49:41.130129 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.130105 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q288c\" (UniqueName: \"kubernetes.io/projected/c3ba442d-4651-44c3-b29d-14d536188970-kube-api-access-q288c\") pod \"ingress-canary-bgs6z\" (UID: \"c3ba442d-4651-44c3-b29d-14d536188970\") " pod="openshift-ingress-canary/ingress-canary-bgs6z" Jul 26 17:49:41.130251 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.130234 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz6js\" (UniqueName: \"kubernetes.io/projected/1f1f2371-6ad2-4065-929a-d4e87827fa52-kube-api-access-fz6js\") pod \"dns-default-5q7q7\" (UID: \"1f1f2371-6ad2-4065-929a-d4e87827fa52\") " pod="openshift-dns/dns-default-5q7q7" Jul 26 17:49:41.130336 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.130312 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-bound-sa-token\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.130542 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.130526 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zfkl\" (UniqueName: \"kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-kube-api-access-2zfkl\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.221917 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.221877 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bgs6z" Jul 26 17:49:41.433684 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.433490 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:49:41.433776 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.433539 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:49:41.437255 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.437123 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zs84s\"" Jul 26 17:49:41.438224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.437389 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nb78b\"" Jul 26 17:49:41.438224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.437497 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 26 17:49:41.438224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.437576 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 26 17:49:41.438224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.437817 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 26 17:49:41.511054 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.511028 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bgs6z"] Jul 26 17:49:41.514287 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:41.514257 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3ba442d_4651_44c3_b29d_14d536188970.slice/crio-d8061ba63fd8860412579c52162407e63f440eee877317ee474d69b238dd940d WatchSource:0}: Error finding container d8061ba63fd8860412579c52162407e63f440eee877317ee474d69b238dd940d: Status 404 returned error can't find the container with id d8061ba63fd8860412579c52162407e63f440eee877317ee474d69b238dd940d Jul 26 17:49:41.598824 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.598788 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bgs6z" event={"ID":"c3ba442d-4651-44c3-b29d-14d536188970","Type":"ContainerStarted","Data":"d8061ba63fd8860412579c52162407e63f440eee877317ee474d69b238dd940d"} Jul 26 17:49:41.601406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.601374 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5cpxq" event={"ID":"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0","Type":"ContainerStarted","Data":"4f0cc323beae75a21eab3a855fd745efe169add298776818ceb75f57cb3ef440"} Jul 26 17:49:41.624530 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.624505 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-tls\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:41.624640 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:41.624564 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f1f2371-6ad2-4065-929a-d4e87827fa52-metrics-tls\") pod \"dns-default-5q7q7\" (UID: \"1f1f2371-6ad2-4065-929a-d4e87827fa52\") " pod="openshift-dns/dns-default-5q7q7" Jul 26 17:49:41.624764 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:41.624742 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 26 17:49:41.624838 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:41.624823 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f1f2371-6ad2-4065-929a-d4e87827fa52-metrics-tls podName:1f1f2371-6ad2-4065-929a-d4e87827fa52 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:42.624804536 +0000 UTC m=+36.577460206 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1f1f2371-6ad2-4065-929a-d4e87827fa52-metrics-tls") pod "dns-default-5q7q7" (UID: "1f1f2371-6ad2-4065-929a-d4e87827fa52") : secret "dns-default-metrics-tls" not found Jul 26 17:49:41.625200 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:41.625185 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 26 17:49:41.625200 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:41.625202 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68cdbc4cd7-qj2k4: secret "image-registry-tls" not found Jul 26 17:49:41.625287 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:41.625250 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-tls podName:0b76c942-35c5-49e9-a469-db6ccacf23ff nodeName:}" failed. No retries permitted until 2026-07-26 17:49:42.625235877 +0000 UTC m=+36.577891550 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-tls") pod "image-registry-68cdbc4cd7-qj2k4" (UID: "0b76c942-35c5-49e9-a469-db6ccacf23ff") : secret "image-registry-tls" not found Jul 26 17:49:42.185691 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:42.185638 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:42.258282 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:42.258247 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:42.314795 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:42.314748 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:42.432843 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:42.432760 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:49:42.435603 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:42.435575 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 26 17:49:42.607605 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:42.607569 2559 generic.go:356] "Generic (PLEG): container finished" podID="0c0e8dd9-a868-44f9-bec5-aa0aaae878d0" containerID="4f0cc323beae75a21eab3a855fd745efe169add298776818ceb75f57cb3ef440" exitCode=0 Jul 26 17:49:42.607802 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:42.607651 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5cpxq" event={"ID":"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0","Type":"ContainerDied","Data":"4f0cc323beae75a21eab3a855fd745efe169add298776818ceb75f57cb3ef440"} Jul 26 17:49:42.633217 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:42.633189 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-tls\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:42.633344 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:42.633316 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 26 17:49:42.633344 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:42.633339 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68cdbc4cd7-qj2k4: secret "image-registry-tls" not found Jul 26 17:49:42.633473 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:42.633344 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f1f2371-6ad2-4065-929a-d4e87827fa52-metrics-tls\") pod \"dns-default-5q7q7\" (UID: \"1f1f2371-6ad2-4065-929a-d4e87827fa52\") " pod="openshift-dns/dns-default-5q7q7" Jul 26 17:49:42.633473 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:42.633391 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-tls podName:0b76c942-35c5-49e9-a469-db6ccacf23ff nodeName:}" failed. No retries permitted until 2026-07-26 17:49:44.633375466 +0000 UTC m=+38.586031148 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-tls") pod "image-registry-68cdbc4cd7-qj2k4" (UID: "0b76c942-35c5-49e9-a469-db6ccacf23ff") : secret "image-registry-tls" not found Jul 26 17:49:42.633473 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:42.633464 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 26 17:49:42.633615 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:42.633529 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f1f2371-6ad2-4065-929a-d4e87827fa52-metrics-tls podName:1f1f2371-6ad2-4065-929a-d4e87827fa52 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:44.633511233 +0000 UTC m=+38.586166900 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1f1f2371-6ad2-4065-929a-d4e87827fa52-metrics-tls") pod "dns-default-5q7q7" (UID: "1f1f2371-6ad2-4065-929a-d4e87827fa52") : secret "dns-default-metrics-tls" not found Jul 26 17:49:42.674129 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:42.674095 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:42.774967 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:42.774933 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:43.369416 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:43.369378 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:43.471048 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:43.471007 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:43.612897 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:43.612868 2559 generic.go:356] "Generic (PLEG): container finished" podID="0c0e8dd9-a868-44f9-bec5-aa0aaae878d0" containerID="05a31d7b7627a3c97dba081d88187d08dbcbd8a0c99230e49f0ece36d52e2ee9" exitCode=0 Jul 26 17:49:43.613070 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:43.612908 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5cpxq" event={"ID":"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0","Type":"ContainerDied","Data":"05a31d7b7627a3c97dba081d88187d08dbcbd8a0c99230e49f0ece36d52e2ee9"} Jul 26 17:49:44.619371 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:44.619115 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5cpxq" event={"ID":"0c0e8dd9-a868-44f9-bec5-aa0aaae878d0","Type":"ContainerStarted","Data":"c0c73f9bc80f58ca8db17128b871fbe8348ebf547c903fa9825b8596a244fac2"} Jul 26 17:49:44.620591 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:44.620568 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bgs6z" event={"ID":"c3ba442d-4651-44c3-b29d-14d536188970","Type":"ContainerStarted","Data":"bff395afc8f55e26feef065501b2093312a1b8afba8c9388a0221a573ea79942"} Jul 26 17:49:44.643642 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:44.643594 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5cpxq" podStartSLOduration=9.334913915 podStartE2EDuration="38.643578128s" totalImagesPullingTime="29.308664213s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:06 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:44.641895849 +0000 UTC m=+38.594551558" watchObservedRunningTime="2026-07-26 17:49:44.643578128 +0000 UTC m=+38.596233817" Jul 26 17:49:44.651231 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:44.651206 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-tls\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:44.651324 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:44.651253 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f1f2371-6ad2-4065-929a-d4e87827fa52-metrics-tls\") pod \"dns-default-5q7q7\" (UID: \"1f1f2371-6ad2-4065-929a-d4e87827fa52\") " pod="openshift-dns/dns-default-5q7q7" Jul 26 17:49:44.651387 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:44.651359 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 26 17:49:44.651387 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:44.651359 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 26 17:49:44.651387 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:44.651379 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68cdbc4cd7-qj2k4: secret "image-registry-tls" not found Jul 26 17:49:44.651493 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:44.651408 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f1f2371-6ad2-4065-929a-d4e87827fa52-metrics-tls podName:1f1f2371-6ad2-4065-929a-d4e87827fa52 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:48.651394983 +0000 UTC m=+42.604050648 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1f1f2371-6ad2-4065-929a-d4e87827fa52-metrics-tls") pod "dns-default-5q7q7" (UID: "1f1f2371-6ad2-4065-929a-d4e87827fa52") : secret "dns-default-metrics-tls" not found Jul 26 17:49:44.651493 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:44.651436 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-tls podName:0b76c942-35c5-49e9-a469-db6ccacf23ff nodeName:}" failed. No retries permitted until 2026-07-26 17:49:48.651421836 +0000 UTC m=+42.604077502 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-tls") pod "image-registry-68cdbc4cd7-qj2k4" (UID: "0b76c942-35c5-49e9-a469-db6ccacf23ff") : secret "image-registry-tls" not found Jul 26 17:49:44.657948 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:44.657890 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bgs6z" podStartSLOduration=2.627361454 podStartE2EDuration="4.657871441s" totalImagesPullingTime="2.030509987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:49:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:44.65754349 +0000 UTC m=+38.610199198" watchObservedRunningTime="2026-07-26 17:49:44.657871441 +0000 UTC m=+38.610527127" Jul 26 17:49:44.870724 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:44.870623 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:44.924933 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:44.924903 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:46.230289 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:46.230253 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:46.331294 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:46.331258 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:47.912865 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:47.912834 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-vzm9r"] Jul 26 17:49:47.913332 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:47.913306 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:47.953048 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:47.953011 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-vzm9r"] Jul 26 17:49:47.953205 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:47.953130 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-vzm9r" Jul 26 17:49:47.955709 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:47.955662 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 26 17:49:47.956632 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:47.956616 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 26 17:49:47.956717 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:47.956694 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-kbj98\"" Jul 26 17:49:47.974853 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:47.974824 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:48.035994 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:48.035960 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:48.075756 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:48.075724 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq46g\" (UniqueName: \"kubernetes.io/projected/650616e9-345b-43fd-aab6-895e5af929d5-kube-api-access-rq46g\") pod \"migrator-6865c9967-vzm9r\" (UID: \"650616e9-345b-43fd-aab6-895e5af929d5\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-vzm9r" Jul 26 17:49:48.176522 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:48.176432 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq46g\" (UniqueName: \"kubernetes.io/projected/650616e9-345b-43fd-aab6-895e5af929d5-kube-api-access-rq46g\") pod \"migrator-6865c9967-vzm9r\" (UID: \"650616e9-345b-43fd-aab6-895e5af929d5\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-vzm9r" Jul 26 17:49:48.191810 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:48.191777 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq46g\" (UniqueName: \"kubernetes.io/projected/650616e9-345b-43fd-aab6-895e5af929d5-kube-api-access-rq46g\") pod \"migrator-6865c9967-vzm9r\" (UID: \"650616e9-345b-43fd-aab6-895e5af929d5\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-vzm9r" Jul 26 17:49:48.262721 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:48.262684 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-vzm9r" Jul 26 17:49:48.315298 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:48.315264 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:48.370831 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:48.370797 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:48.401395 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:48.401227 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-vzm9r"] Jul 26 17:49:48.403898 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:48.403872 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod650616e9_345b_43fd_aab6_895e5af929d5.slice/crio-bb07e21475515215f9b32e9b53e9417725ed257ba19f0b038814ba03a4b6a109 WatchSource:0}: Error finding container bb07e21475515215f9b32e9b53e9417725ed257ba19f0b038814ba03a4b6a109: Status 404 returned error can't find the container with id bb07e21475515215f9b32e9b53e9417725ed257ba19f0b038814ba03a4b6a109 Jul 26 17:49:48.631000 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:48.630917 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-vzm9r" event={"ID":"650616e9-345b-43fd-aab6-895e5af929d5","Type":"ContainerStarted","Data":"bb07e21475515215f9b32e9b53e9417725ed257ba19f0b038814ba03a4b6a109"} Jul 26 17:49:48.679635 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:48.679601 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-tls\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:48.679730 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:48.679656 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f1f2371-6ad2-4065-929a-d4e87827fa52-metrics-tls\") pod \"dns-default-5q7q7\" (UID: \"1f1f2371-6ad2-4065-929a-d4e87827fa52\") " pod="openshift-dns/dns-default-5q7q7" Jul 26 17:49:48.679794 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:48.679781 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 26 17:49:48.679853 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:48.679844 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f1f2371-6ad2-4065-929a-d4e87827fa52-metrics-tls podName:1f1f2371-6ad2-4065-929a-d4e87827fa52 nodeName:}" failed. No retries permitted until 2026-07-26 17:49:56.679830765 +0000 UTC m=+50.632486431 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1f1f2371-6ad2-4065-929a-d4e87827fa52-metrics-tls") pod "dns-default-5q7q7" (UID: "1f1f2371-6ad2-4065-929a-d4e87827fa52") : secret "dns-default-metrics-tls" not found Jul 26 17:49:48.679896 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:48.679784 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 26 17:49:48.679896 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:48.679883 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68cdbc4cd7-qj2k4: secret "image-registry-tls" not found Jul 26 17:49:48.679960 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:48.679930 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-tls podName:0b76c942-35c5-49e9-a469-db6ccacf23ff nodeName:}" failed. No retries permitted until 2026-07-26 17:49:56.679917054 +0000 UTC m=+50.632572720 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-tls") pod "image-registry-68cdbc4cd7-qj2k4" (UID: "0b76c942-35c5-49e9-a469-db6ccacf23ff") : secret "image-registry-tls" not found Jul 26 17:49:49.024082 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:49.024002 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:49.126220 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:49.126171 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:50.623065 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.623031 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-fswc5"] Jul 26 17:49:50.638239 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.638207 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-vzm9r" event={"ID":"650616e9-345b-43fd-aab6-895e5af929d5","Type":"ContainerStarted","Data":"b94c2767170d8712a981e7111b7d58c67edd20aca1c1f3245bfff4fea648510a"} Jul 26 17:49:50.638239 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.638246 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-fswc5"] Jul 26 17:49:50.638465 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.638265 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-vzm9r" event={"ID":"650616e9-345b-43fd-aab6-895e5af929d5","Type":"ContainerStarted","Data":"69819feb6544ddde55912fc6f9e56373683ba78b9746192b787fe38593d41ddb"} Jul 26 17:49:50.638465 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.638361 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-fswc5" Jul 26 17:49:50.641107 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.641082 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 26 17:49:50.641243 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.641085 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 26 17:49:50.641243 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.641083 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 26 17:49:50.642085 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.642066 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 26 17:49:50.642198 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.642066 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-cxkhm\"" Jul 26 17:49:50.659206 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.659176 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:50.667150 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.667109 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-vzm9r" podStartSLOduration=1.9786722650000002 podStartE2EDuration="3.667093714s" totalImagesPullingTime="1.688421449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:49:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:50.666504706 +0000 UTC m=+44.619160393" watchObservedRunningTime="2026-07-26 17:49:50.667093714 +0000 UTC m=+44.619749401" Jul 26 17:49:50.725276 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.725246 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:50.796306 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.796264 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a4a152ff-7d7f-4c7e-9eae-3c83c9d5428b-signing-cabundle\") pod \"service-ca-658d7b7c78-fswc5\" (UID: \"a4a152ff-7d7f-4c7e-9eae-3c83c9d5428b\") " pod="openshift-service-ca/service-ca-658d7b7c78-fswc5" Jul 26 17:49:50.796470 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.796325 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a4a152ff-7d7f-4c7e-9eae-3c83c9d5428b-signing-key\") pod \"service-ca-658d7b7c78-fswc5\" (UID: \"a4a152ff-7d7f-4c7e-9eae-3c83c9d5428b\") " pod="openshift-service-ca/service-ca-658d7b7c78-fswc5" Jul 26 17:49:50.796470 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.796396 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv54j\" (UniqueName: \"kubernetes.io/projected/a4a152ff-7d7f-4c7e-9eae-3c83c9d5428b-kube-api-access-qv54j\") pod \"service-ca-658d7b7c78-fswc5\" (UID: \"a4a152ff-7d7f-4c7e-9eae-3c83c9d5428b\") " pod="openshift-service-ca/service-ca-658d7b7c78-fswc5" Jul 26 17:49:50.826920 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.826888 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:50.897480 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.897408 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a4a152ff-7d7f-4c7e-9eae-3c83c9d5428b-signing-key\") pod \"service-ca-658d7b7c78-fswc5\" (UID: \"a4a152ff-7d7f-4c7e-9eae-3c83c9d5428b\") " pod="openshift-service-ca/service-ca-658d7b7c78-fswc5" Jul 26 17:49:50.897480 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.897444 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv54j\" (UniqueName: \"kubernetes.io/projected/a4a152ff-7d7f-4c7e-9eae-3c83c9d5428b-kube-api-access-qv54j\") pod \"service-ca-658d7b7c78-fswc5\" (UID: \"a4a152ff-7d7f-4c7e-9eae-3c83c9d5428b\") " pod="openshift-service-ca/service-ca-658d7b7c78-fswc5" Jul 26 17:49:50.897637 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.897522 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a4a152ff-7d7f-4c7e-9eae-3c83c9d5428b-signing-cabundle\") pod \"service-ca-658d7b7c78-fswc5\" (UID: \"a4a152ff-7d7f-4c7e-9eae-3c83c9d5428b\") " pod="openshift-service-ca/service-ca-658d7b7c78-fswc5" Jul 26 17:49:50.898134 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.898116 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a4a152ff-7d7f-4c7e-9eae-3c83c9d5428b-signing-cabundle\") pod \"service-ca-658d7b7c78-fswc5\" (UID: \"a4a152ff-7d7f-4c7e-9eae-3c83c9d5428b\") " pod="openshift-service-ca/service-ca-658d7b7c78-fswc5" Jul 26 17:49:50.899791 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.899766 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a4a152ff-7d7f-4c7e-9eae-3c83c9d5428b-signing-key\") pod \"service-ca-658d7b7c78-fswc5\" (UID: \"a4a152ff-7d7f-4c7e-9eae-3c83c9d5428b\") " pod="openshift-service-ca/service-ca-658d7b7c78-fswc5" Jul 26 17:49:50.906148 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.906128 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv54j\" (UniqueName: \"kubernetes.io/projected/a4a152ff-7d7f-4c7e-9eae-3c83c9d5428b-kube-api-access-qv54j\") pod \"service-ca-658d7b7c78-fswc5\" (UID: \"a4a152ff-7d7f-4c7e-9eae-3c83c9d5428b\") " pod="openshift-service-ca/service-ca-658d7b7c78-fswc5" Jul 26 17:49:50.948045 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:50.948006 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-fswc5" Jul 26 17:49:51.090628 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:51.090595 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-fswc5"] Jul 26 17:49:51.093305 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:49:51.093277 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4a152ff_7d7f_4c7e_9eae_3c83c9d5428b.slice/crio-37ece0f98d4c0231e2936a000aa4409982635e2e84596ca98c356576b456f498 WatchSource:0}: Error finding container 37ece0f98d4c0231e2936a000aa4409982635e2e84596ca98c356576b456f498: Status 404 returned error can't find the container with id 37ece0f98d4c0231e2936a000aa4409982635e2e84596ca98c356576b456f498 Jul 26 17:49:51.516765 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:51.516725 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:51.642602 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:51.642566 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-fswc5" event={"ID":"a4a152ff-7d7f-4c7e-9eae-3c83c9d5428b","Type":"ContainerStarted","Data":"37ece0f98d4c0231e2936a000aa4409982635e2e84596ca98c356576b456f498"} Jul 26 17:49:53.653272 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:53.653238 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-fswc5" event={"ID":"a4a152ff-7d7f-4c7e-9eae-3c83c9d5428b","Type":"ContainerStarted","Data":"57857cced9ccf7152f063666ed1fa07642d396715de2ee4996b385af71dd0694"} Jul 26 17:49:53.669253 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:53.669197 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-fswc5" podStartSLOduration=1.312526037 podStartE2EDuration="3.669181776s" totalImagesPullingTime="2.356655739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:49:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:49:53.668514933 +0000 UTC m=+47.621170622" watchObservedRunningTime="2026-07-26 17:49:53.669181776 +0000 UTC m=+47.621837487" Jul 26 17:49:54.094017 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:54.093975 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:55.006031 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:55.005934 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:55.146709 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:55.146650 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:55.301179 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:55.301086 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:55.391019 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:55.390983 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:55.526074 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:55.526042 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:55.707045 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:55.707007 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:56.501494 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:56.501460 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:56.745726 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:56.745665 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-tls\") pod \"image-registry-68cdbc4cd7-qj2k4\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:49:56.745905 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:56.745830 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 26 17:49:56.745905 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:56.745833 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f1f2371-6ad2-4065-929a-d4e87827fa52-metrics-tls\") pod \"dns-default-5q7q7\" (UID: \"1f1f2371-6ad2-4065-929a-d4e87827fa52\") " pod="openshift-dns/dns-default-5q7q7" Jul 26 17:49:56.745905 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:56.745848 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-68cdbc4cd7-qj2k4: secret "image-registry-tls" not found Jul 26 17:49:56.745905 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:56.745899 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-tls podName:0b76c942-35c5-49e9-a469-db6ccacf23ff nodeName:}" failed. No retries permitted until 2026-07-26 17:50:12.745883268 +0000 UTC m=+66.698538933 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-tls") pod "image-registry-68cdbc4cd7-qj2k4" (UID: "0b76c942-35c5-49e9-a469-db6ccacf23ff") : secret "image-registry-tls" not found Jul 26 17:49:56.746071 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:56.745964 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 26 17:49:56.746071 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:49:56.746017 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f1f2371-6ad2-4065-929a-d4e87827fa52-metrics-tls podName:1f1f2371-6ad2-4065-929a-d4e87827fa52 nodeName:}" failed. No retries permitted until 2026-07-26 17:50:12.746001817 +0000 UTC m=+66.698657490 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1f1f2371-6ad2-4065-929a-d4e87827fa52-metrics-tls") pod "dns-default-5q7q7" (UID: "1f1f2371-6ad2-4065-929a-d4e87827fa52") : secret "dns-default-metrics-tls" not found Jul 26 17:49:56.899029 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:56.898932 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:57.292907 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:57.292874 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:57.494170 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:57.494136 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:57.921718 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:57.921687 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:58.496483 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:58.496449 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:58.714941 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:58.714901 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:58.905174 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:58.905085 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:59.095380 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:59.095344 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:59.296075 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:59.296036 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:59.495287 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:59.495249 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:59.563395 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:59.563309 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:49:59.919314 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:49:59.919221 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:00.700483 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:00.700450 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:01.097413 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:01.097373 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:01.215051 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:01.215018 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:01.370865 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:01.370780 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:01.498816 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:01.498782 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:01.903870 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:01.903840 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:02.500204 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:02.500170 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:02.573248 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:02.573216 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:02.923184 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:02.923100 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:03.297269 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:03.297233 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:03.697366 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:03.697282 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:04.098868 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:04.098839 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:04.493595 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:04.493562 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:04.898374 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:04.898283 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:05.091572 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:05.091536 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:05.296391 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:05.296360 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:06.598531 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:06.598505 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dc9sh" Jul 26 17:50:09.422764 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.422728 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-92gfp"] Jul 26 17:50:09.426048 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.426025 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-92gfp" Jul 26 17:50:09.430532 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.430511 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 26 17:50:09.430646 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.430598 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 26 17:50:09.431365 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.431348 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-tsbr9"] Jul 26 17:50:09.431877 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.431848 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-d5n8w\"" Jul 26 17:50:09.431972 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.431887 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 26 17:50:09.431972 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.431898 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 26 17:50:09.433436 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.433420 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-tsbr9" Jul 26 17:50:09.436041 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.436021 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 26 17:50:09.436609 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.436595 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-p8w9s\"" Jul 26 17:50:09.436858 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.436844 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 26 17:50:09.436952 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.436913 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 26 17:50:09.437738 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.437725 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 26 17:50:09.446310 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.446286 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d77795ff6-wrpzt"] Jul 26 17:50:09.447167 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.447142 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 26 17:50:09.448373 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.448356 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-92gfp"] Jul 26 17:50:09.448447 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.448394 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99"] Jul 26 17:50:09.448496 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.448468 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d77795ff6-wrpzt" Jul 26 17:50:09.450876 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.450854 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-tsbr9"] Jul 26 17:50:09.450987 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.450974 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" Jul 26 17:50:09.451625 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.451597 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 26 17:50:09.451742 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.451633 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 26 17:50:09.451835 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.451818 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-8289v\"" Jul 26 17:50:09.451926 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.451910 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 26 17:50:09.452088 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.452066 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 26 17:50:09.453921 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.453904 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 26 17:50:09.454329 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.454315 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 26 17:50:09.454909 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.454894 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 26 17:50:09.456246 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.456206 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 26 17:50:09.464836 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.464816 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d77795ff6-wrpzt"] Jul 26 17:50:09.465525 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.465506 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-68cdbc4cd7-qj2k4"] Jul 26 17:50:09.465738 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:50:09.465717 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" podUID="0b76c942-35c5-49e9-a469-db6ccacf23ff" Jul 26 17:50:09.476920 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.476895 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99"] Jul 26 17:50:09.492289 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.492256 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:09.527384 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.527343 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7896856b5b-8tdxc"] Jul 26 17:50:09.529807 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.529791 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.535616 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.535592 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8c35763-fccd-40d0-8109-a5b681b9f675-tmp\") pod \"insights-operator-5885fdb759-tsbr9\" (UID: \"f8c35763-fccd-40d0-8109-a5b681b9f675\") " pod="openshift-insights/insights-operator-5885fdb759-tsbr9" Jul 26 17:50:09.535761 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.535637 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/24a734d6-b903-4cfc-b9dd-d98bb9c336f3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-92gfp\" (UID: \"24a734d6-b903-4cfc-b9dd-d98bb9c336f3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-92gfp" Jul 26 17:50:09.535761 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.535665 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2vl4\" (UniqueName: \"kubernetes.io/projected/24a734d6-b903-4cfc-b9dd-d98bb9c336f3-kube-api-access-l2vl4\") pod \"cluster-monitoring-operator-548b489cb8-92gfp\" (UID: \"24a734d6-b903-4cfc-b9dd-d98bb9c336f3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-92gfp" Jul 26 17:50:09.535761 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.535740 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccd617b5-1da3-4719-8a66-bdf789a3a338-additional-service-ca\") pod \"cluster-proxy-proxy-agent-558d757d97-ddx99\" (UID: \"ccd617b5-1da3-4719-8a66-bdf789a3a338\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" Jul 26 17:50:09.535884 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.535821 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fbeb3542-1c1a-4142-afc0-2311c754f185-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5d77795ff6-wrpzt\" (UID: \"fbeb3542-1c1a-4142-afc0-2311c754f185\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d77795ff6-wrpzt" Jul 26 17:50:09.535884 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.535865 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8c35763-fccd-40d0-8109-a5b681b9f675-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-tsbr9\" (UID: \"f8c35763-fccd-40d0-8109-a5b681b9f675\") " pod="openshift-insights/insights-operator-5885fdb759-tsbr9" Jul 26 17:50:09.535989 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.535886 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f8c35763-fccd-40d0-8109-a5b681b9f675-snapshots\") pod \"insights-operator-5885fdb759-tsbr9\" (UID: \"f8c35763-fccd-40d0-8109-a5b681b9f675\") " pod="openshift-insights/insights-operator-5885fdb759-tsbr9" Jul 26 17:50:09.535989 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.535928 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tt6v\" (UniqueName: \"kubernetes.io/projected/fbeb3542-1c1a-4142-afc0-2311c754f185-kube-api-access-6tt6v\") pod \"managed-serviceaccount-addon-agent-5d77795ff6-wrpzt\" (UID: \"fbeb3542-1c1a-4142-afc0-2311c754f185\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d77795ff6-wrpzt" Jul 26 17:50:09.535989 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.535964 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ccd617b5-1da3-4719-8a66-bdf789a3a338-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-558d757d97-ddx99\" (UID: \"ccd617b5-1da3-4719-8a66-bdf789a3a338\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" Jul 26 17:50:09.536098 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.535988 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/24a734d6-b903-4cfc-b9dd-d98bb9c336f3-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-92gfp\" (UID: \"24a734d6-b903-4cfc-b9dd-d98bb9c336f3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-92gfp" Jul 26 17:50:09.536098 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.536034 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ccd617b5-1da3-4719-8a66-bdf789a3a338-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-558d757d97-ddx99\" (UID: \"ccd617b5-1da3-4719-8a66-bdf789a3a338\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" Jul 26 17:50:09.536098 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.536071 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ccd617b5-1da3-4719-8a66-bdf789a3a338-hub\") pod \"cluster-proxy-proxy-agent-558d757d97-ddx99\" (UID: \"ccd617b5-1da3-4719-8a66-bdf789a3a338\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" Jul 26 17:50:09.536098 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.536092 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f8c35763-fccd-40d0-8109-a5b681b9f675-serving-cert\") pod \"insights-operator-5885fdb759-tsbr9\" (UID: \"f8c35763-fccd-40d0-8109-a5b681b9f675\") " pod="openshift-insights/insights-operator-5885fdb759-tsbr9" Jul 26 17:50:09.536230 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.536107 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57hr9\" (UniqueName: \"kubernetes.io/projected/ccd617b5-1da3-4719-8a66-bdf789a3a338-kube-api-access-57hr9\") pod \"cluster-proxy-proxy-agent-558d757d97-ddx99\" (UID: \"ccd617b5-1da3-4719-8a66-bdf789a3a338\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" Jul 26 17:50:09.536230 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.536132 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8c35763-fccd-40d0-8109-a5b681b9f675-service-ca-bundle\") pod \"insights-operator-5885fdb759-tsbr9\" (UID: \"f8c35763-fccd-40d0-8109-a5b681b9f675\") " pod="openshift-insights/insights-operator-5885fdb759-tsbr9" Jul 26 17:50:09.536230 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.536162 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5vkq\" (UniqueName: \"kubernetes.io/projected/f8c35763-fccd-40d0-8109-a5b681b9f675-kube-api-access-g5vkq\") pod \"insights-operator-5885fdb759-tsbr9\" (UID: \"f8c35763-fccd-40d0-8109-a5b681b9f675\") " pod="openshift-insights/insights-operator-5885fdb759-tsbr9" Jul 26 17:50:09.536230 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.536185 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ccd617b5-1da3-4719-8a66-bdf789a3a338-ca\") pod \"cluster-proxy-proxy-agent-558d757d97-ddx99\" (UID: \"ccd617b5-1da3-4719-8a66-bdf789a3a338\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" Jul 26 17:50:09.545622 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.545600 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7896856b5b-8tdxc"] Jul 26 17:50:09.590739 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.590702 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:09.637464 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.637434 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/24a734d6-b903-4cfc-b9dd-d98bb9c336f3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-92gfp\" (UID: \"24a734d6-b903-4cfc-b9dd-d98bb9c336f3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-92gfp" Jul 26 17:50:09.637464 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.637464 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2vl4\" (UniqueName: \"kubernetes.io/projected/24a734d6-b903-4cfc-b9dd-d98bb9c336f3-kube-api-access-l2vl4\") pod \"cluster-monitoring-operator-548b489cb8-92gfp\" (UID: \"24a734d6-b903-4cfc-b9dd-d98bb9c336f3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-92gfp" Jul 26 17:50:09.637658 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.637485 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccd617b5-1da3-4719-8a66-bdf789a3a338-additional-service-ca\") pod \"cluster-proxy-proxy-agent-558d757d97-ddx99\" (UID: \"ccd617b5-1da3-4719-8a66-bdf789a3a338\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" Jul 26 17:50:09.637658 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.637513 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fbeb3542-1c1a-4142-afc0-2311c754f185-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5d77795ff6-wrpzt\" (UID: \"fbeb3542-1c1a-4142-afc0-2311c754f185\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d77795ff6-wrpzt" Jul 26 17:50:09.637658 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.637597 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8c35763-fccd-40d0-8109-a5b681b9f675-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-tsbr9\" (UID: \"f8c35763-fccd-40d0-8109-a5b681b9f675\") " pod="openshift-insights/insights-operator-5885fdb759-tsbr9" Jul 26 17:50:09.637658 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.637636 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/037b4a96-1fc7-46a7-beca-05ab6867c352-registry-certificates\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.637854 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.637780 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f8c35763-fccd-40d0-8109-a5b681b9f675-snapshots\") pod \"insights-operator-5885fdb759-tsbr9\" (UID: \"f8c35763-fccd-40d0-8109-a5b681b9f675\") " pod="openshift-insights/insights-operator-5885fdb759-tsbr9" Jul 26 17:50:09.637854 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.637826 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/037b4a96-1fc7-46a7-beca-05ab6867c352-installation-pull-secrets\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.637955 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.637854 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tt6v\" (UniqueName: \"kubernetes.io/projected/fbeb3542-1c1a-4142-afc0-2311c754f185-kube-api-access-6tt6v\") pod \"managed-serviceaccount-addon-agent-5d77795ff6-wrpzt\" (UID: \"fbeb3542-1c1a-4142-afc0-2311c754f185\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d77795ff6-wrpzt" Jul 26 17:50:09.637955 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.637891 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ccd617b5-1da3-4719-8a66-bdf789a3a338-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-558d757d97-ddx99\" (UID: \"ccd617b5-1da3-4719-8a66-bdf789a3a338\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" Jul 26 17:50:09.637955 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.637916 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/24a734d6-b903-4cfc-b9dd-d98bb9c336f3-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-92gfp\" (UID: \"24a734d6-b903-4cfc-b9dd-d98bb9c336f3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-92gfp" Jul 26 17:50:09.638075 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.637956 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ccd617b5-1da3-4719-8a66-bdf789a3a338-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-558d757d97-ddx99\" (UID: \"ccd617b5-1da3-4719-8a66-bdf789a3a338\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" Jul 26 17:50:09.638075 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.637983 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/037b4a96-1fc7-46a7-beca-05ab6867c352-image-registry-private-configuration\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.638075 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.638012 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ccd617b5-1da3-4719-8a66-bdf789a3a338-hub\") pod \"cluster-proxy-proxy-agent-558d757d97-ddx99\" (UID: \"ccd617b5-1da3-4719-8a66-bdf789a3a338\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" Jul 26 17:50:09.638075 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.638041 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f8c35763-fccd-40d0-8109-a5b681b9f675-serving-cert\") pod \"insights-operator-5885fdb759-tsbr9\" (UID: \"f8c35763-fccd-40d0-8109-a5b681b9f675\") " pod="openshift-insights/insights-operator-5885fdb759-tsbr9" Jul 26 17:50:09.638075 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.638064 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57hr9\" (UniqueName: \"kubernetes.io/projected/ccd617b5-1da3-4719-8a66-bdf789a3a338-kube-api-access-57hr9\") pod \"cluster-proxy-proxy-agent-558d757d97-ddx99\" (UID: \"ccd617b5-1da3-4719-8a66-bdf789a3a338\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" Jul 26 17:50:09.638313 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.638091 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8c35763-fccd-40d0-8109-a5b681b9f675-service-ca-bundle\") pod \"insights-operator-5885fdb759-tsbr9\" (UID: \"f8c35763-fccd-40d0-8109-a5b681b9f675\") " pod="openshift-insights/insights-operator-5885fdb759-tsbr9" Jul 26 17:50:09.638313 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.638119 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5vkq\" (UniqueName: \"kubernetes.io/projected/f8c35763-fccd-40d0-8109-a5b681b9f675-kube-api-access-g5vkq\") pod \"insights-operator-5885fdb759-tsbr9\" (UID: \"f8c35763-fccd-40d0-8109-a5b681b9f675\") " pod="openshift-insights/insights-operator-5885fdb759-tsbr9" Jul 26 17:50:09.638313 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.638144 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ccd617b5-1da3-4719-8a66-bdf789a3a338-ca\") pod \"cluster-proxy-proxy-agent-558d757d97-ddx99\" (UID: \"ccd617b5-1da3-4719-8a66-bdf789a3a338\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" Jul 26 17:50:09.638313 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.638176 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7br8\" (UniqueName: \"kubernetes.io/projected/037b4a96-1fc7-46a7-beca-05ab6867c352-kube-api-access-r7br8\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.638313 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.638206 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/037b4a96-1fc7-46a7-beca-05ab6867c352-trusted-ca\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.638313 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.638228 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/037b4a96-1fc7-46a7-beca-05ab6867c352-bound-sa-token\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.638313 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.638265 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/037b4a96-1fc7-46a7-beca-05ab6867c352-registry-tls\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.638313 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.638304 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8c35763-fccd-40d0-8109-a5b681b9f675-tmp\") pod \"insights-operator-5885fdb759-tsbr9\" (UID: \"f8c35763-fccd-40d0-8109-a5b681b9f675\") " pod="openshift-insights/insights-operator-5885fdb759-tsbr9" Jul 26 17:50:09.638313 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.638304 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccd617b5-1da3-4719-8a66-bdf789a3a338-additional-service-ca\") pod \"cluster-proxy-proxy-agent-558d757d97-ddx99\" (UID: \"ccd617b5-1da3-4719-8a66-bdf789a3a338\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" Jul 26 17:50:09.638794 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.638327 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/037b4a96-1fc7-46a7-beca-05ab6867c352-ca-trust-extracted\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.638794 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.638461 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f8c35763-fccd-40d0-8109-a5b681b9f675-snapshots\") pod \"insights-operator-5885fdb759-tsbr9\" (UID: \"f8c35763-fccd-40d0-8109-a5b681b9f675\") " pod="openshift-insights/insights-operator-5885fdb759-tsbr9" Jul 26 17:50:09.638794 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.638687 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8c35763-fccd-40d0-8109-a5b681b9f675-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-tsbr9\" (UID: \"f8c35763-fccd-40d0-8109-a5b681b9f675\") " pod="openshift-insights/insights-operator-5885fdb759-tsbr9" Jul 26 17:50:09.639658 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.639569 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/24a734d6-b903-4cfc-b9dd-d98bb9c336f3-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-92gfp\" (UID: \"24a734d6-b903-4cfc-b9dd-d98bb9c336f3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-92gfp" Jul 26 17:50:09.640077 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.640036 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8c35763-fccd-40d0-8109-a5b681b9f675-tmp\") pod \"insights-operator-5885fdb759-tsbr9\" (UID: \"f8c35763-fccd-40d0-8109-a5b681b9f675\") " pod="openshift-insights/insights-operator-5885fdb759-tsbr9" Jul 26 17:50:09.640174 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.640115 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8c35763-fccd-40d0-8109-a5b681b9f675-service-ca-bundle\") pod \"insights-operator-5885fdb759-tsbr9\" (UID: \"f8c35763-fccd-40d0-8109-a5b681b9f675\") " pod="openshift-insights/insights-operator-5885fdb759-tsbr9" Jul 26 17:50:09.640704 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.640664 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/24a734d6-b903-4cfc-b9dd-d98bb9c336f3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-92gfp\" (UID: \"24a734d6-b903-4cfc-b9dd-d98bb9c336f3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-92gfp" Jul 26 17:50:09.640816 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.640778 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fbeb3542-1c1a-4142-afc0-2311c754f185-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5d77795ff6-wrpzt\" (UID: \"fbeb3542-1c1a-4142-afc0-2311c754f185\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d77795ff6-wrpzt" Jul 26 17:50:09.640816 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.640785 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ccd617b5-1da3-4719-8a66-bdf789a3a338-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-558d757d97-ddx99\" (UID: \"ccd617b5-1da3-4719-8a66-bdf789a3a338\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" Jul 26 17:50:09.640979 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.640959 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ccd617b5-1da3-4719-8a66-bdf789a3a338-ca\") pod \"cluster-proxy-proxy-agent-558d757d97-ddx99\" (UID: \"ccd617b5-1da3-4719-8a66-bdf789a3a338\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" Jul 26 17:50:09.641979 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.641959 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ccd617b5-1da3-4719-8a66-bdf789a3a338-hub\") pod \"cluster-proxy-proxy-agent-558d757d97-ddx99\" (UID: \"ccd617b5-1da3-4719-8a66-bdf789a3a338\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" Jul 26 17:50:09.642485 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.642464 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f8c35763-fccd-40d0-8109-a5b681b9f675-serving-cert\") pod \"insights-operator-5885fdb759-tsbr9\" (UID: \"f8c35763-fccd-40d0-8109-a5b681b9f675\") " pod="openshift-insights/insights-operator-5885fdb759-tsbr9" Jul 26 17:50:09.642564 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.642540 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ccd617b5-1da3-4719-8a66-bdf789a3a338-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-558d757d97-ddx99\" (UID: \"ccd617b5-1da3-4719-8a66-bdf789a3a338\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" Jul 26 17:50:09.648692 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.648643 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57hr9\" (UniqueName: \"kubernetes.io/projected/ccd617b5-1da3-4719-8a66-bdf789a3a338-kube-api-access-57hr9\") pod \"cluster-proxy-proxy-agent-558d757d97-ddx99\" (UID: \"ccd617b5-1da3-4719-8a66-bdf789a3a338\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" Jul 26 17:50:09.648781 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.648653 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2vl4\" (UniqueName: \"kubernetes.io/projected/24a734d6-b903-4cfc-b9dd-d98bb9c336f3-kube-api-access-l2vl4\") pod \"cluster-monitoring-operator-548b489cb8-92gfp\" (UID: \"24a734d6-b903-4cfc-b9dd-d98bb9c336f3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-92gfp" Jul 26 17:50:09.649158 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.649136 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tt6v\" (UniqueName: \"kubernetes.io/projected/fbeb3542-1c1a-4142-afc0-2311c754f185-kube-api-access-6tt6v\") pod \"managed-serviceaccount-addon-agent-5d77795ff6-wrpzt\" (UID: \"fbeb3542-1c1a-4142-afc0-2311c754f185\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d77795ff6-wrpzt" Jul 26 17:50:09.649254 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.649235 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5vkq\" (UniqueName: \"kubernetes.io/projected/f8c35763-fccd-40d0-8109-a5b681b9f675-kube-api-access-g5vkq\") pod \"insights-operator-5885fdb759-tsbr9\" (UID: \"f8c35763-fccd-40d0-8109-a5b681b9f675\") " pod="openshift-insights/insights-operator-5885fdb759-tsbr9" Jul 26 17:50:09.693503 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.693424 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:50:09.700432 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.700403 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:50:09.728332 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.728300 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:09.735590 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.735568 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-92gfp" Jul 26 17:50:09.738832 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.738816 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0b76c942-35c5-49e9-a469-db6ccacf23ff-ca-trust-extracted\") pod \"0b76c942-35c5-49e9-a469-db6ccacf23ff\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " Jul 26 17:50:09.738895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.738850 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0b76c942-35c5-49e9-a469-db6ccacf23ff-installation-pull-secrets\") pod \"0b76c942-35c5-49e9-a469-db6ccacf23ff\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " Jul 26 17:50:09.738895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.738879 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0b76c942-35c5-49e9-a469-db6ccacf23ff-image-registry-private-configuration\") pod \"0b76c942-35c5-49e9-a469-db6ccacf23ff\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " Jul 26 17:50:09.739032 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.739013 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0b76c942-35c5-49e9-a469-db6ccacf23ff-trusted-ca\") pod \"0b76c942-35c5-49e9-a469-db6ccacf23ff\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " Jul 26 17:50:09.739087 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.739060 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zfkl\" (UniqueName: \"kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-kube-api-access-2zfkl\") pod \"0b76c942-35c5-49e9-a469-db6ccacf23ff\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " Jul 26 17:50:09.739170 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.739142 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b76c942-35c5-49e9-a469-db6ccacf23ff-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0b76c942-35c5-49e9-a469-db6ccacf23ff" (UID: "0b76c942-35c5-49e9-a469-db6ccacf23ff"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 17:50:09.739306 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.739188 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-bound-sa-token\") pod \"0b76c942-35c5-49e9-a469-db6ccacf23ff\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " Jul 26 17:50:09.739306 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.739228 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-certificates\") pod \"0b76c942-35c5-49e9-a469-db6ccacf23ff\" (UID: \"0b76c942-35c5-49e9-a469-db6ccacf23ff\") " Jul 26 17:50:09.739418 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.739325 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/037b4a96-1fc7-46a7-beca-05ab6867c352-registry-certificates\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.739418 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.739380 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/037b4a96-1fc7-46a7-beca-05ab6867c352-installation-pull-secrets\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.739512 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.739419 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b76c942-35c5-49e9-a469-db6ccacf23ff-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0b76c942-35c5-49e9-a469-db6ccacf23ff" (UID: "0b76c942-35c5-49e9-a469-db6ccacf23ff"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 17:50:09.739512 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.739431 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/037b4a96-1fc7-46a7-beca-05ab6867c352-image-registry-private-configuration\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.739640 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.739611 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0b76c942-35c5-49e9-a469-db6ccacf23ff" (UID: "0b76c942-35c5-49e9-a469-db6ccacf23ff"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 17:50:09.739753 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.739665 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7br8\" (UniqueName: \"kubernetes.io/projected/037b4a96-1fc7-46a7-beca-05ab6867c352-kube-api-access-r7br8\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.739823 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.739781 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/037b4a96-1fc7-46a7-beca-05ab6867c352-trusted-ca\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.739823 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.739813 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/037b4a96-1fc7-46a7-beca-05ab6867c352-bound-sa-token\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.739925 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.739844 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/037b4a96-1fc7-46a7-beca-05ab6867c352-registry-tls\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.739925 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.739888 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/037b4a96-1fc7-46a7-beca-05ab6867c352-ca-trust-extracted\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.740026 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.740005 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0b76c942-35c5-49e9-a469-db6ccacf23ff-trusted-ca\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 17:50:09.740078 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.740023 2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-certificates\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 17:50:09.740078 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.740039 2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0b76c942-35c5-49e9-a469-db6ccacf23ff-ca-trust-extracted\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 17:50:09.740440 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.740325 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/037b4a96-1fc7-46a7-beca-05ab6867c352-registry-certificates\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.740440 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.740336 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/037b4a96-1fc7-46a7-beca-05ab6867c352-ca-trust-extracted\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.741561 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.741210 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/037b4a96-1fc7-46a7-beca-05ab6867c352-trusted-ca\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.741657 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.741630 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-kube-api-access-2zfkl" (OuterVolumeSpecName: "kube-api-access-2zfkl") pod "0b76c942-35c5-49e9-a469-db6ccacf23ff" (UID: "0b76c942-35c5-49e9-a469-db6ccacf23ff"). InnerVolumeSpecName "kube-api-access-2zfkl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 17:50:09.741971 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.741787 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-tsbr9" Jul 26 17:50:09.742113 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.742045 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0b76c942-35c5-49e9-a469-db6ccacf23ff" (UID: "0b76c942-35c5-49e9-a469-db6ccacf23ff"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 17:50:09.742366 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.742346 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/037b4a96-1fc7-46a7-beca-05ab6867c352-image-registry-private-configuration\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.742366 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.742346 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b76c942-35c5-49e9-a469-db6ccacf23ff-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0b76c942-35c5-49e9-a469-db6ccacf23ff" (UID: "0b76c942-35c5-49e9-a469-db6ccacf23ff"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 26 17:50:09.742524 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.742503 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b76c942-35c5-49e9-a469-db6ccacf23ff-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0b76c942-35c5-49e9-a469-db6ccacf23ff" (UID: "0b76c942-35c5-49e9-a469-db6ccacf23ff"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 26 17:50:09.742975 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.742959 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/037b4a96-1fc7-46a7-beca-05ab6867c352-installation-pull-secrets\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.743906 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.743888 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/037b4a96-1fc7-46a7-beca-05ab6867c352-registry-tls\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.751259 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.751208 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/037b4a96-1fc7-46a7-beca-05ab6867c352-bound-sa-token\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.751464 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.751438 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7br8\" (UniqueName: \"kubernetes.io/projected/037b4a96-1fc7-46a7-beca-05ab6867c352-kube-api-access-r7br8\") pod \"image-registry-7896856b5b-8tdxc\" (UID: \"037b4a96-1fc7-46a7-beca-05ab6867c352\") " pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.774445 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.774367 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d77795ff6-wrpzt" Jul 26 17:50:09.779123 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.779093 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" Jul 26 17:50:09.809336 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.809278 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:09.841309 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.841217 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zfkl\" (UniqueName: \"kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-kube-api-access-2zfkl\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 17:50:09.841309 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.841247 2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-bound-sa-token\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 17:50:09.841309 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.841266 2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0b76c942-35c5-49e9-a469-db6ccacf23ff-installation-pull-secrets\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 17:50:09.841309 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.841280 2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0b76c942-35c5-49e9-a469-db6ccacf23ff-image-registry-private-configuration\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 17:50:09.865758 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.865717 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2hr9p\"" Jul 26 17:50:09.873118 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.873089 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:09.907787 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.907708 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:09.908300 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.908276 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-tsbr9"] Jul 26 17:50:09.911814 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:50:09.911778 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8c35763_fccd_40d0_8109_a5b681b9f675.slice/crio-4663334ae90cc07798ab44ae446902ae3c93b8f2578c98bdf41aea9f54da93ed WatchSource:0}: Error finding container 4663334ae90cc07798ab44ae446902ae3c93b8f2578c98bdf41aea9f54da93ed: Status 404 returned error can't find the container with id 4663334ae90cc07798ab44ae446902ae3c93b8f2578c98bdf41aea9f54da93ed Jul 26 17:50:09.929574 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.929496 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-92gfp"] Jul 26 17:50:09.939881 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:50:09.939834 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24a734d6_b903_4cfc_b9dd_d98bb9c336f3.slice/crio-49914da0a4e958e1b8f6fd3129dc67e98fac60c71d2d202e4576e431b5938203 WatchSource:0}: Error finding container 49914da0a4e958e1b8f6fd3129dc67e98fac60c71d2d202e4576e431b5938203: Status 404 returned error can't find the container with id 49914da0a4e958e1b8f6fd3129dc67e98fac60c71d2d202e4576e431b5938203 Jul 26 17:50:09.965535 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.965510 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d77795ff6-wrpzt"] Jul 26 17:50:09.968158 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:50:09.968015 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfbeb3542_1c1a_4142_afc0_2311c754f185.slice/crio-93cc120f9c32f43c924c0247c04d91d2a59e7e7ea24edf61f4065fe7525ede11 WatchSource:0}: Error finding container 93cc120f9c32f43c924c0247c04d91d2a59e7e7ea24edf61f4065fe7525ede11: Status 404 returned error can't find the container with id 93cc120f9c32f43c924c0247c04d91d2a59e7e7ea24edf61f4065fe7525ede11 Jul 26 17:50:09.981301 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:50:09.981258 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podccd617b5_1da3_4719_8a66_bdf789a3a338.slice/crio-05057a1eaede05ce08dbb6afa8c7882c8b7cc888dbe90964a0a93384472f43a2 WatchSource:0}: Error finding container 05057a1eaede05ce08dbb6afa8c7882c8b7cc888dbe90964a0a93384472f43a2: Status 404 returned error can't find the container with id 05057a1eaede05ce08dbb6afa8c7882c8b7cc888dbe90964a0a93384472f43a2 Jul 26 17:50:09.983102 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.983080 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99"] Jul 26 17:50:09.991299 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:09.991267 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:10.019831 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:10.019801 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7896856b5b-8tdxc"] Jul 26 17:50:10.022580 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:50:10.022555 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod037b4a96_1fc7_46a7_beca_05ab6867c352.slice/crio-5f20fa42e83f3445a8daeb5e46be4c1e540b508b488b0fa4f1b5e046a79af5dd WatchSource:0}: Error finding container 5f20fa42e83f3445a8daeb5e46be4c1e540b508b488b0fa4f1b5e046a79af5dd: Status 404 returned error can't find the container with id 5f20fa42e83f3445a8daeb5e46be4c1e540b508b488b0fa4f1b5e046a79af5dd Jul 26 17:50:10.307541 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:10.307502 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:10.395098 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:10.395061 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:10.502999 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:10.502952 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:10.608063 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:10.607979 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:10.702691 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:10.702581 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-92gfp" event={"ID":"24a734d6-b903-4cfc-b9dd-d98bb9c336f3","Type":"ContainerStarted","Data":"49914da0a4e958e1b8f6fd3129dc67e98fac60c71d2d202e4576e431b5938203"} Jul 26 17:50:10.711613 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:10.711541 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d77795ff6-wrpzt" event={"ID":"fbeb3542-1c1a-4142-afc0-2311c754f185","Type":"ContainerStarted","Data":"93cc120f9c32f43c924c0247c04d91d2a59e7e7ea24edf61f4065fe7525ede11"} Jul 26 17:50:10.714348 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:10.714260 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" event={"ID":"ccd617b5-1da3-4719-8a66-bdf789a3a338","Type":"ContainerStarted","Data":"05057a1eaede05ce08dbb6afa8c7882c8b7cc888dbe90964a0a93384472f43a2"} Jul 26 17:50:10.717899 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:10.717823 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-tsbr9" event={"ID":"f8c35763-fccd-40d0-8109-a5b681b9f675","Type":"ContainerStarted","Data":"4663334ae90cc07798ab44ae446902ae3c93b8f2578c98bdf41aea9f54da93ed"} Jul 26 17:50:10.728167 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:10.724423 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68cdbc4cd7-qj2k4" Jul 26 17:50:10.728167 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:10.725118 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" event={"ID":"037b4a96-1fc7-46a7-beca-05ab6867c352","Type":"ContainerStarted","Data":"c4f066275389279d363371ac06b1e82e5e4b7e5e221f57fae762362d1ed3dd13"} Jul 26 17:50:10.728167 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:10.725196 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" event={"ID":"037b4a96-1fc7-46a7-beca-05ab6867c352","Type":"ContainerStarted","Data":"5f20fa42e83f3445a8daeb5e46be4c1e540b508b488b0fa4f1b5e046a79af5dd"} Jul 26 17:50:10.728167 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:10.725406 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:10.751004 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:10.749921 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" podStartSLOduration=1.7498999149999999 podStartE2EDuration="1.749899915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:50:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:10.749220061 +0000 UTC m=+64.701875772" watchObservedRunningTime="2026-07-26 17:50:10.749899915 +0000 UTC m=+64.702555603" Jul 26 17:50:10.781087 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:10.781033 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-68cdbc4cd7-qj2k4"] Jul 26 17:50:10.783720 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:10.783697 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-68cdbc4cd7-qj2k4"] Jul 26 17:50:10.849019 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:10.848967 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0b76c942-35c5-49e9-a469-db6ccacf23ff-registry-tls\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 17:50:10.872286 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:10.872204 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:10.939069 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:10.938918 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:11.656203 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:11.656170 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:11.757863 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:11.757825 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:12.063982 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:12.063946 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:12.160126 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:12.160086 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret\") pod \"global-pull-secret-syncer-fgwwx\" (UID: \"92dc3c1b-48ce-4b80-858c-5421f0bcc2f2\") " pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:50:12.163030 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:12.163006 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 26 17:50:12.165991 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:12.165953 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:12.173106 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:12.173084 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92dc3c1b-48ce-4b80-858c-5421f0bcc2f2-original-pull-secret\") pod \"global-pull-secret-syncer-fgwwx\" (UID: \"92dc3c1b-48ce-4b80-858c-5421f0bcc2f2\") " pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:50:12.261546 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:12.261509 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs\") pod \"network-metrics-daemon-v5q7r\" (UID: \"b6d9352c-2170-418f-9836-89640e78d1e1\") " pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:50:12.264386 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:12.264359 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 26 17:50:12.274752 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:12.274721 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6d9352c-2170-418f-9836-89640e78d1e1-metrics-certs\") pod \"network-metrics-daemon-v5q7r\" (UID: \"b6d9352c-2170-418f-9836-89640e78d1e1\") " pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:50:12.359270 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:12.359192 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zs84s\"" Jul 26 17:50:12.362730 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:12.362697 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5wz7\" (UniqueName: \"kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7\") pod \"network-check-target-2sjzw\" (UID: \"bab84b92-d4aa-422d-a364-195365d32cfb\") " pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:50:12.365555 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:12.365532 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 26 17:50:12.366539 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:12.366513 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5q7r" Jul 26 17:50:12.376427 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:12.376408 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 26 17:50:12.387223 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:12.387200 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5wz7\" (UniqueName: \"kubernetes.io/projected/bab84b92-d4aa-422d-a364-195365d32cfb-kube-api-access-x5wz7\") pod \"network-check-target-2sjzw\" (UID: \"bab84b92-d4aa-422d-a364-195365d32cfb\") " pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:50:12.437601 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:12.437565 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b76c942-35c5-49e9-a469-db6ccacf23ff" path="/var/lib/kubelet/pods/0b76c942-35c5-49e9-a469-db6ccacf23ff/volumes" Jul 26 17:50:12.444293 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:12.444255 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fgwwx" Jul 26 17:50:12.652877 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:12.652798 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nb78b\"" Jul 26 17:50:12.660932 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:12.660902 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:50:12.766261 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:12.766223 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f1f2371-6ad2-4065-929a-d4e87827fa52-metrics-tls\") pod \"dns-default-5q7q7\" (UID: \"1f1f2371-6ad2-4065-929a-d4e87827fa52\") " pod="openshift-dns/dns-default-5q7q7" Jul 26 17:50:12.768857 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:12.768819 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f1f2371-6ad2-4065-929a-d4e87827fa52-metrics-tls\") pod \"dns-default-5q7q7\" (UID: \"1f1f2371-6ad2-4065-929a-d4e87827fa52\") " pod="openshift-dns/dns-default-5q7q7" Jul 26 17:50:12.879345 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:12.879306 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:13.016090 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:13.016052 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jb7dx\"" Jul 26 17:50:13.024042 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:13.024006 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5q7q7" Jul 26 17:50:13.273068 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:13.272982 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:13.374168 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:13.374130 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:13.474276 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:13.474228 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:14.553474 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:14.553445 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v5q7r"] Jul 26 17:50:14.558296 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:50:14.558251 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d9352c_2170_418f_9836_89640e78d1e1.slice/crio-56a0f1c1ae373558ab8bfdbdae9b4252468ee225fe02c8acfe4a9fd0d3e4c432 WatchSource:0}: Error finding container 56a0f1c1ae373558ab8bfdbdae9b4252468ee225fe02c8acfe4a9fd0d3e4c432: Status 404 returned error can't find the container with id 56a0f1c1ae373558ab8bfdbdae9b4252468ee225fe02c8acfe4a9fd0d3e4c432 Jul 26 17:50:14.606494 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:14.606416 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fgwwx"] Jul 26 17:50:14.612393 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:50:14.612359 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92dc3c1b_48ce_4b80_858c_5421f0bcc2f2.slice/crio-f7150edabf736f3ddfbb0ca67e2f90c65a4c9fc224433782ccccd7c37c0c0598 WatchSource:0}: Error finding container f7150edabf736f3ddfbb0ca67e2f90c65a4c9fc224433782ccccd7c37c0c0598: Status 404 returned error can't find the container with id f7150edabf736f3ddfbb0ca67e2f90c65a4c9fc224433782ccccd7c37c0c0598 Jul 26 17:50:14.745994 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:14.745931 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d77795ff6-wrpzt" event={"ID":"fbeb3542-1c1a-4142-afc0-2311c754f185","Type":"ContainerStarted","Data":"b2bf774ea2efd4efb8abbccefb4f230a53d7c6e978bc685e9f97469017482ec6"} Jul 26 17:50:14.747829 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:14.747797 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v5q7r" event={"ID":"b6d9352c-2170-418f-9836-89640e78d1e1","Type":"ContainerStarted","Data":"56a0f1c1ae373558ab8bfdbdae9b4252468ee225fe02c8acfe4a9fd0d3e4c432"} Jul 26 17:50:14.749738 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:14.749702 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-tsbr9" event={"ID":"f8c35763-fccd-40d0-8109-a5b681b9f675","Type":"ContainerStarted","Data":"07713a12412e66bff0ab4f66a54a720675a8f7e42051ab26f8174c726bc320d7"} Jul 26 17:50:14.751624 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:14.751603 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-92gfp" event={"ID":"24a734d6-b903-4cfc-b9dd-d98bb9c336f3","Type":"ContainerStarted","Data":"3338d83554fd4d0c3c09856c22d0528e5cc11492fd4013446183aa3938702b40"} Jul 26 17:50:14.752905 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:14.752882 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fgwwx" event={"ID":"92dc3c1b-48ce-4b80-858c-5421f0bcc2f2","Type":"ContainerStarted","Data":"f7150edabf736f3ddfbb0ca67e2f90c65a4c9fc224433782ccccd7c37c0c0598"} Jul 26 17:50:14.764689 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:14.764615 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d77795ff6-wrpzt" podStartSLOduration=1.354102028 podStartE2EDuration="5.764597868s" totalImagesPullingTime="4.41049584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:50:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:14.763504493 +0000 UTC m=+68.716160182" watchObservedRunningTime="2026-07-26 17:50:14.764597868 +0000 UTC m=+68.717253577" Jul 26 17:50:14.776169 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:14.776138 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2sjzw"] Jul 26 17:50:14.780376 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:50:14.780348 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbab84b92_d4aa_422d_a364_195365d32cfb.slice/crio-fd4082923e18ee9b6b1dc7f8ee3259c244dfa86d2239da2969ab6235f8384dd4 WatchSource:0}: Error finding container fd4082923e18ee9b6b1dc7f8ee3259c244dfa86d2239da2969ab6235f8384dd4: Status 404 returned error can't find the container with id fd4082923e18ee9b6b1dc7f8ee3259c244dfa86d2239da2969ab6235f8384dd4 Jul 26 17:50:14.782631 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:14.782603 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5q7q7"] Jul 26 17:50:14.786410 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:14.786358 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-92gfp" podStartSLOduration=1.344156726 podStartE2EDuration="5.786342298s" totalImagesPullingTime="4.442185572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:50:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:14.786081454 +0000 UTC m=+68.738737142" watchObservedRunningTime="2026-07-26 17:50:14.786342298 +0000 UTC m=+68.738997981" Jul 26 17:50:14.786648 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:50:14.786617 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f1f2371_6ad2_4065_929a_d4e87827fa52.slice/crio-bc5d74b238e8c4da4d0aebd1f2bbdaf66125700b46568ac16e8bc34cd0dcbb36 WatchSource:0}: Error finding container bc5d74b238e8c4da4d0aebd1f2bbdaf66125700b46568ac16e8bc34cd0dcbb36: Status 404 returned error can't find the container with id bc5d74b238e8c4da4d0aebd1f2bbdaf66125700b46568ac16e8bc34cd0dcbb36 Jul 26 17:50:14.809031 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:14.808983 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-tsbr9" podStartSLOduration=1.33950179 podStartE2EDuration="5.808967578s" totalImagesPullingTime="4.469465788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:50:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:14.808084569 +0000 UTC m=+68.760740257" watchObservedRunningTime="2026-07-26 17:50:14.808967578 +0000 UTC m=+68.761623244" Jul 26 17:50:14.845227 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:14.845196 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:14.948764 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:14.948726 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:15.056651 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.056543 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:15.175826 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.175684 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:15.239007 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.238951 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:15.524956 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.524901 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:15.710603 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.710493 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gfwst"] Jul 26 17:50:15.718784 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.718760 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:15.721346 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.721272 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gfwst"] Jul 26 17:50:15.723722 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.722495 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 26 17:50:15.723722 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.722573 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dk9zk\"" Jul 26 17:50:15.723722 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.723443 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 26 17:50:15.725759 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.725734 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:15.758603 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.758571 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2sjzw" event={"ID":"bab84b92-d4aa-422d-a364-195365d32cfb","Type":"ContainerStarted","Data":"fd4082923e18ee9b6b1dc7f8ee3259c244dfa86d2239da2969ab6235f8384dd4"} Jul 26 17:50:15.761602 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.761574 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5q7q7" event={"ID":"1f1f2371-6ad2-4065-929a-d4e87827fa52","Type":"ContainerStarted","Data":"bc5d74b238e8c4da4d0aebd1f2bbdaf66125700b46568ac16e8bc34cd0dcbb36"} Jul 26 17:50:15.793608 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.793528 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn9ld\" (UniqueName: \"kubernetes.io/projected/67f55fe3-f055-4bd0-8728-bcb5fa86db98-kube-api-access-vn9ld\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:15.793800 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.793613 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67f55fe3-f055-4bd0-8728-bcb5fa86db98-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:15.793800 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.793718 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/67f55fe3-f055-4bd0-8728-bcb5fa86db98-data-volume\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:15.793930 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.793809 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67f55fe3-f055-4bd0-8728-bcb5fa86db98-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:15.793930 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.793850 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/67f55fe3-f055-4bd0-8728-bcb5fa86db98-crio-socket\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:15.808444 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.808382 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:15.897562 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.896924 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67f55fe3-f055-4bd0-8728-bcb5fa86db98-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:15.897562 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.896984 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/67f55fe3-f055-4bd0-8728-bcb5fa86db98-crio-socket\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:15.897562 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:50:15.897026 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 26 17:50:15.897562 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.897032 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn9ld\" (UniqueName: \"kubernetes.io/projected/67f55fe3-f055-4bd0-8728-bcb5fa86db98-kube-api-access-vn9ld\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:15.897562 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.897082 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67f55fe3-f055-4bd0-8728-bcb5fa86db98-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:15.897562 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:50:15.897105 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67f55fe3-f055-4bd0-8728-bcb5fa86db98-kube-rbac-proxy-cm podName:67f55fe3-f055-4bd0-8728-bcb5fa86db98 nodeName:}" failed. No retries permitted until 2026-07-26 17:50:16.397085923 +0000 UTC m=+70.349741593 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67f55fe3-f055-4bd0-8728-bcb5fa86db98-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gfwst" (UID: "67f55fe3-f055-4bd0-8728-bcb5fa86db98") : configmap "kube-rbac-proxy" not found Jul 26 17:50:15.897562 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.897151 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/67f55fe3-f055-4bd0-8728-bcb5fa86db98-data-volume\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:15.897562 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:50:15.897180 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 26 17:50:15.897562 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:50:15.897233 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67f55fe3-f055-4bd0-8728-bcb5fa86db98-insights-runtime-extractor-tls podName:67f55fe3-f055-4bd0-8728-bcb5fa86db98 nodeName:}" failed. No retries permitted until 2026-07-26 17:50:16.397215487 +0000 UTC m=+70.349871160 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67f55fe3-f055-4bd0-8728-bcb5fa86db98-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gfwst" (UID: "67f55fe3-f055-4bd0-8728-bcb5fa86db98") : secret "insights-runtime-extractor-tls" not found Jul 26 17:50:15.897562 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.897281 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/67f55fe3-f055-4bd0-8728-bcb5fa86db98-crio-socket\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:15.897562 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.897493 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/67f55fe3-f055-4bd0-8728-bcb5fa86db98-data-volume\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:15.908835 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:15.908782 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn9ld\" (UniqueName: \"kubernetes.io/projected/67f55fe3-f055-4bd0-8728-bcb5fa86db98-kube-api-access-vn9ld\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:16.401580 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:16.401493 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67f55fe3-f055-4bd0-8728-bcb5fa86db98-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:16.401580 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:16.401570 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67f55fe3-f055-4bd0-8728-bcb5fa86db98-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:16.401890 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:50:16.401689 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 26 17:50:16.401890 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:50:16.401755 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67f55fe3-f055-4bd0-8728-bcb5fa86db98-kube-rbac-proxy-cm podName:67f55fe3-f055-4bd0-8728-bcb5fa86db98 nodeName:}" failed. No retries permitted until 2026-07-26 17:50:17.401735846 +0000 UTC m=+71.354391519 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67f55fe3-f055-4bd0-8728-bcb5fa86db98-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gfwst" (UID: "67f55fe3-f055-4bd0-8728-bcb5fa86db98") : configmap "kube-rbac-proxy" not found Jul 26 17:50:16.401890 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:50:16.401804 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 26 17:50:16.401890 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:50:16.401867 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67f55fe3-f055-4bd0-8728-bcb5fa86db98-insights-runtime-extractor-tls podName:67f55fe3-f055-4bd0-8728-bcb5fa86db98 nodeName:}" failed. No retries permitted until 2026-07-26 17:50:17.40185081 +0000 UTC m=+71.354506493 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67f55fe3-f055-4bd0-8728-bcb5fa86db98-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gfwst" (UID: "67f55fe3-f055-4bd0-8728-bcb5fa86db98") : secret "insights-runtime-extractor-tls" not found Jul 26 17:50:16.769417 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:16.769168 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v5q7r" event={"ID":"b6d9352c-2170-418f-9836-89640e78d1e1","Type":"ContainerStarted","Data":"8d4edf6e2302aca2af67803b61c189b449d22490cc4c426dcaddac69c230273d"} Jul 26 17:50:16.769417 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:16.769273 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v5q7r" event={"ID":"b6d9352c-2170-418f-9836-89640e78d1e1","Type":"ContainerStarted","Data":"63f0c8b77a6da5cfcac357dacf77f2708f523c60238009a8a9256e0014f69b23"} Jul 26 17:50:16.788725 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:16.788380 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-v5q7r" podStartSLOduration=69.509359114 podStartE2EDuration="1m10.788334485s" totalImagesPullingTime="1.278975371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:16.78710692 +0000 UTC m=+70.739762608" watchObservedRunningTime="2026-07-26 17:50:16.788334485 +0000 UTC m=+70.740990172" Jul 26 17:50:16.909897 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:16.909863 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:17.032544 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:17.032154 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:17.411360 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:17.411249 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:17.412479 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:17.411922 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67f55fe3-f055-4bd0-8728-bcb5fa86db98-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:17.412479 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:17.411997 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67f55fe3-f055-4bd0-8728-bcb5fa86db98-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:17.412479 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:50:17.412099 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 26 17:50:17.412479 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:50:17.412162 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67f55fe3-f055-4bd0-8728-bcb5fa86db98-kube-rbac-proxy-cm podName:67f55fe3-f055-4bd0-8728-bcb5fa86db98 nodeName:}" failed. No retries permitted until 2026-07-26 17:50:19.412144394 +0000 UTC m=+73.364800061 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67f55fe3-f055-4bd0-8728-bcb5fa86db98-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gfwst" (UID: "67f55fe3-f055-4bd0-8728-bcb5fa86db98") : configmap "kube-rbac-proxy" not found Jul 26 17:50:17.412479 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:50:17.412249 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 26 17:50:17.412479 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:50:17.412311 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67f55fe3-f055-4bd0-8728-bcb5fa86db98-insights-runtime-extractor-tls podName:67f55fe3-f055-4bd0-8728-bcb5fa86db98 nodeName:}" failed. No retries permitted until 2026-07-26 17:50:19.412298779 +0000 UTC m=+73.364954460 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67f55fe3-f055-4bd0-8728-bcb5fa86db98-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gfwst" (UID: "67f55fe3-f055-4bd0-8728-bcb5fa86db98") : secret "insights-runtime-extractor-tls" not found Jul 26 17:50:17.514061 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:17.513966 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:17.515372 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:17.515347 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2575c_e0cf3a10-8a54-45ad-aafa-bb7175208585/dns-node-resolver/0.log" Jul 26 17:50:17.777018 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:17.776934 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5q7q7" event={"ID":"1f1f2371-6ad2-4065-929a-d4e87827fa52","Type":"ContainerStarted","Data":"c0771bc70101439f8e3553a702ca933ee579998e1282f991c6e0d86c48152577"} Jul 26 17:50:17.777018 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:17.777001 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5q7q7" event={"ID":"1f1f2371-6ad2-4065-929a-d4e87827fa52","Type":"ContainerStarted","Data":"64340bd784539b23937561c7c805ba1a4497eed410e5481f7262e7c4c8221104"} Jul 26 17:50:17.777541 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:17.777113 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5q7q7" Jul 26 17:50:17.795329 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:17.795179 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5q7q7" podStartSLOduration=35.627742773 podStartE2EDuration="37.79515631s" totalImagesPullingTime="2.167413537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:49:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:17.794175283 +0000 UTC m=+71.746830991" watchObservedRunningTime="2026-07-26 17:50:17.79515631 +0000 UTC m=+71.747812002" Jul 26 17:50:18.116152 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:18.116012 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7896856b5b-8tdxc_037b4a96-1fc7-46a7-beca-05ab6867c352/registry/0.log" Jul 26 17:50:18.514190 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:18.514159 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bh5vf_a5ef698f-5999-4902-9fe6-57b185586841/node-ca/0.log" Jul 26 17:50:19.314225 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:19.314197 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bgs6z_c3ba442d-4651-44c3-b29d-14d536188970/serve-healthcheck-canary/0.log" Jul 26 17:50:19.432262 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:19.432224 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67f55fe3-f055-4bd0-8728-bcb5fa86db98-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:19.432453 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:19.432308 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67f55fe3-f055-4bd0-8728-bcb5fa86db98-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:19.432453 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:50:19.432391 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 26 17:50:19.432453 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:50:19.432439 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 26 17:50:19.432613 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:50:19.432484 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67f55fe3-f055-4bd0-8728-bcb5fa86db98-insights-runtime-extractor-tls podName:67f55fe3-f055-4bd0-8728-bcb5fa86db98 nodeName:}" failed. No retries permitted until 2026-07-26 17:50:23.432462995 +0000 UTC m=+77.385118666 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67f55fe3-f055-4bd0-8728-bcb5fa86db98-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gfwst" (UID: "67f55fe3-f055-4bd0-8728-bcb5fa86db98") : secret "insights-runtime-extractor-tls" not found Jul 26 17:50:19.432613 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:50:19.432506 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67f55fe3-f055-4bd0-8728-bcb5fa86db98-kube-rbac-proxy-cm podName:67f55fe3-f055-4bd0-8728-bcb5fa86db98 nodeName:}" failed. No retries permitted until 2026-07-26 17:50:23.432495976 +0000 UTC m=+77.385151645 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67f55fe3-f055-4bd0-8728-bcb5fa86db98-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gfwst" (UID: "67f55fe3-f055-4bd0-8728-bcb5fa86db98") : configmap "kube-rbac-proxy" not found Jul 26 17:50:20.056630 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:20.056597 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:20.123448 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:20.123406 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:20.388817 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:20.388728 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:21.794360 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:21.794328 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fgwwx" event={"ID":"92dc3c1b-48ce-4b80-858c-5421f0bcc2f2","Type":"ContainerStarted","Data":"8d9cfb16887fe0026c1fc2c663c9a84b8cab69ec5af0c2c4b4f2617bfb5f02e4"} Jul 26 17:50:21.795870 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:21.795847 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2sjzw" event={"ID":"bab84b92-d4aa-422d-a364-195365d32cfb","Type":"ContainerStarted","Data":"4f7a8781b43912c1f51e2e0fe2eb535c2e631575b9216d0634a2172b398c3c70"} Jul 26 17:50:21.795998 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:21.795949 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:50:21.797806 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:21.797783 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" event={"ID":"ccd617b5-1da3-4719-8a66-bdf789a3a338","Type":"ContainerStarted","Data":"769d82ebbd462e60d4316cccb23ae535e619003c112166d46d21808f8648b3ec"} Jul 26 17:50:21.797914 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:21.797818 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" event={"ID":"ccd617b5-1da3-4719-8a66-bdf789a3a338","Type":"ContainerStarted","Data":"edec7f510094fbd900ad79d851aa08a7b2785ece0d6effb34b5b35de0e1adb10"} Jul 26 17:50:21.797914 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:21.797836 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" event={"ID":"ccd617b5-1da3-4719-8a66-bdf789a3a338","Type":"ContainerStarted","Data":"e580c87d92635825d6d131c27f4314c19c59c014a94fb12c0caa2d41df5826b3"} Jul 26 17:50:21.819035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:21.818995 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fgwwx" podStartSLOduration=69.260195045 podStartE2EDuration="1m15.818982907s" totalImagesPullingTime="6.558787862s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:21.818357423 +0000 UTC m=+75.771013128" watchObservedRunningTime="2026-07-26 17:50:21.818982907 +0000 UTC m=+75.771638592" Jul 26 17:50:21.870005 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:21.869950 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558d757d97-ddx99" podStartSLOduration=1.677597951 podStartE2EDuration="12.869935215s" totalImagesPullingTime="11.192337264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:50:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:21.869472841 +0000 UTC m=+75.822128530" watchObservedRunningTime="2026-07-26 17:50:21.869935215 +0000 UTC m=+75.822590903" Jul 26 17:50:21.902566 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:21.902507 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2sjzw" podStartSLOduration=69.522445354 podStartE2EDuration="1m15.902490648s" totalImagesPullingTime="6.380045294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:21.901045533 +0000 UTC m=+75.853701256" watchObservedRunningTime="2026-07-26 17:50:21.902490648 +0000 UTC m=+75.855146336" Jul 26 17:50:23.401454 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:23.401418 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:23.470977 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:23.470939 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67f55fe3-f055-4bd0-8728-bcb5fa86db98-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:23.471167 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:23.471008 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67f55fe3-f055-4bd0-8728-bcb5fa86db98-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:23.471167 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:50:23.471095 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 26 17:50:23.471167 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:50:23.471107 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 26 17:50:23.471167 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:50:23.471167 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67f55fe3-f055-4bd0-8728-bcb5fa86db98-kube-rbac-proxy-cm podName:67f55fe3-f055-4bd0-8728-bcb5fa86db98 nodeName:}" failed. No retries permitted until 2026-07-26 17:50:31.471149749 +0000 UTC m=+85.423805429 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67f55fe3-f055-4bd0-8728-bcb5fa86db98-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gfwst" (UID: "67f55fe3-f055-4bd0-8728-bcb5fa86db98") : configmap "kube-rbac-proxy" not found Jul 26 17:50:23.471333 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:50:23.471187 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67f55fe3-f055-4bd0-8728-bcb5fa86db98-insights-runtime-extractor-tls podName:67f55fe3-f055-4bd0-8728-bcb5fa86db98 nodeName:}" failed. No retries permitted until 2026-07-26 17:50:31.471178031 +0000 UTC m=+85.423833700 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67f55fe3-f055-4bd0-8728-bcb5fa86db98-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gfwst" (UID: "67f55fe3-f055-4bd0-8728-bcb5fa86db98") : secret "insights-runtime-extractor-tls" not found Jul 26 17:50:25.201066 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.201033 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:25.386305 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.386260 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:25.511469 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.511425 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5mk9m"] Jul 26 17:50:25.515471 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.515450 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.520051 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.519714 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 26 17:50:25.523687 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.520216 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 26 17:50:25.524695 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.523853 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 26 17:50:25.524695 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.524156 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fs6k7\"" Jul 26 17:50:25.524695 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.524394 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 26 17:50:25.528801 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.528531 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:25.587346 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.587314 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk5r5\" (UniqueName: \"kubernetes.io/projected/324bf551-2df7-4565-a667-33e4f050e53a-kube-api-access-fk5r5\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.587512 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.587351 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/324bf551-2df7-4565-a667-33e4f050e53a-node-exporter-accelerators-collector-config\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.587512 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.587436 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/324bf551-2df7-4565-a667-33e4f050e53a-sys\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.587512 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.587473 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/324bf551-2df7-4565-a667-33e4f050e53a-metrics-client-ca\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.587512 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.587497 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/324bf551-2df7-4565-a667-33e4f050e53a-root\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.587657 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.587515 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/324bf551-2df7-4565-a667-33e4f050e53a-node-exporter-tls\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.587657 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.587559 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/324bf551-2df7-4565-a667-33e4f050e53a-node-exporter-wtmp\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.587657 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.587612 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/324bf551-2df7-4565-a667-33e4f050e53a-node-exporter-textfile\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.587657 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.587643 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/324bf551-2df7-4565-a667-33e4f050e53a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.617659 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.617620 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:25.688991 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.688960 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/324bf551-2df7-4565-a667-33e4f050e53a-node-exporter-wtmp\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.689170 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.689003 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/324bf551-2df7-4565-a667-33e4f050e53a-node-exporter-textfile\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.689170 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.689025 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/324bf551-2df7-4565-a667-33e4f050e53a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.689170 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.689079 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk5r5\" (UniqueName: \"kubernetes.io/projected/324bf551-2df7-4565-a667-33e4f050e53a-kube-api-access-fk5r5\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.689170 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.689106 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/324bf551-2df7-4565-a667-33e4f050e53a-node-exporter-accelerators-collector-config\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.689170 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.689158 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/324bf551-2df7-4565-a667-33e4f050e53a-node-exporter-wtmp\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.689412 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.689163 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/324bf551-2df7-4565-a667-33e4f050e53a-sys\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.689412 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.689212 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/324bf551-2df7-4565-a667-33e4f050e53a-metrics-client-ca\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.689412 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.689212 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/324bf551-2df7-4565-a667-33e4f050e53a-sys\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.689412 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.689247 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/324bf551-2df7-4565-a667-33e4f050e53a-root\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.689412 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.689264 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/324bf551-2df7-4565-a667-33e4f050e53a-node-exporter-tls\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.689412 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.689334 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/324bf551-2df7-4565-a667-33e4f050e53a-root\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.689412 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.689354 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/324bf551-2df7-4565-a667-33e4f050e53a-node-exporter-textfile\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.689808 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.689786 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/324bf551-2df7-4565-a667-33e4f050e53a-metrics-client-ca\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.689872 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.689808 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/324bf551-2df7-4565-a667-33e4f050e53a-node-exporter-accelerators-collector-config\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.691474 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.691456 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/324bf551-2df7-4565-a667-33e4f050e53a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.691579 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.691563 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/324bf551-2df7-4565-a667-33e4f050e53a-node-exporter-tls\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.698117 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.698097 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk5r5\" (UniqueName: \"kubernetes.io/projected/324bf551-2df7-4565-a667-33e4f050e53a-kube-api-access-fk5r5\") pod \"node-exporter-5mk9m\" (UID: \"324bf551-2df7-4565-a667-33e4f050e53a\") " pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.715934 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.715893 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:25.814997 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.814874 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:25.834007 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.833982 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5mk9m" Jul 26 17:50:25.844875 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:50:25.844848 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod324bf551_2df7_4565_a667_33e4f050e53a.slice/crio-f02d5307fe8d687dd2b08caa6e7ebaa7d172c9547fc92af83b060c07b563f22d WatchSource:0}: Error finding container f02d5307fe8d687dd2b08caa6e7ebaa7d172c9547fc92af83b060c07b563f22d: Status 404 returned error can't find the container with id f02d5307fe8d687dd2b08caa6e7ebaa7d172c9547fc92af83b060c07b563f22d Jul 26 17:50:25.917029 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:25.916990 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:26.019535 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:26.019496 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:26.117633 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:26.117553 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:26.386184 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:26.386101 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:26.469488 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:26.469456 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:26.596158 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:26.596096 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:26.702885 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:26.702807 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:26.810333 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:26.810307 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:26.818535 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:26.818508 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5mk9m" event={"ID":"324bf551-2df7-4565-a667-33e4f050e53a","Type":"ContainerStarted","Data":"f02d5307fe8d687dd2b08caa6e7ebaa7d172c9547fc92af83b060c07b563f22d"} Jul 26 17:50:27.785532 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:27.785503 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5q7q7" Jul 26 17:50:27.824603 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:27.824572 2559 generic.go:356] "Generic (PLEG): container finished" podID="324bf551-2df7-4565-a667-33e4f050e53a" containerID="bf9b7247406e226b757303fb44b26a87e8e6391ff6fd4046c9d25688f486f94a" exitCode=0 Jul 26 17:50:27.824775 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:27.824661 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5mk9m" event={"ID":"324bf551-2df7-4565-a667-33e4f050e53a","Type":"ContainerDied","Data":"bf9b7247406e226b757303fb44b26a87e8e6391ff6fd4046c9d25688f486f94a"} Jul 26 17:50:28.367022 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:28.366986 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:28.462992 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:28.462956 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:28.578156 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:28.578119 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:28.652432 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:28.652326 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:28.712796 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:28.712760 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:28.830868 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:28.830837 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5mk9m" event={"ID":"324bf551-2df7-4565-a667-33e4f050e53a","Type":"ContainerStarted","Data":"47a5f08ba700174b3d16c293550d51480e80eca53f8e32859ff68b3c22e2e63a"} Jul 26 17:50:28.831260 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:28.830913 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5mk9m" event={"ID":"324bf551-2df7-4565-a667-33e4f050e53a","Type":"ContainerStarted","Data":"aa5ab2f0eea76935fe081499a497600e0a509c81265d38971fc16b5d982b00e1"} Jul 26 17:50:28.849499 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:28.849451 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5mk9m" podStartSLOduration=2.854200076 podStartE2EDuration="3.8494364s" totalImagesPullingTime="930.673362ms" totalInitContainerRuntime="64.562962ms" isStatefulPod=true podCreationTimestamp="2026-07-26 17:50:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:28.848433649 +0000 UTC m=+82.801089347" watchObservedRunningTime="2026-07-26 17:50:28.8494364 +0000 UTC m=+82.802092088" Jul 26 17:50:29.879130 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:29.879093 2559 patch_prober.go:28] interesting pod/image-registry-7896856b5b-8tdxc container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 26 17:50:29.879538 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:29.879149 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" podUID="037b4a96-1fc7-46a7-beca-05ab6867c352" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 26 17:50:30.221528 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:30.221492 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:30.450663 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:30.450632 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:31.540088 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:31.540049 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67f55fe3-f055-4bd0-8728-bcb5fa86db98-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:31.540532 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:31.540294 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67f55fe3-f055-4bd0-8728-bcb5fa86db98-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:31.540731 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:31.540706 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67f55fe3-f055-4bd0-8728-bcb5fa86db98-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:31.542562 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:31.542541 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67f55fe3-f055-4bd0-8728-bcb5fa86db98-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfwst\" (UID: \"67f55fe3-f055-4bd0-8728-bcb5fa86db98\") " pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:31.634443 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:31.634409 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gfwst" Jul 26 17:50:31.655591 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:31.655556 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:31.732666 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:31.732587 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:31.733136 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:31.733106 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7896856b5b-8tdxc" Jul 26 17:50:31.766365 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:31.766173 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gfwst"] Jul 26 17:50:31.768480 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:50:31.768452 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67f55fe3_f055_4bd0_8728_bcb5fa86db98.slice/crio-fb469cbaa7be22931be73b4b05b8725f0fcd7385c0e9dfeef83cda945fc9d496 WatchSource:0}: Error finding container fb469cbaa7be22931be73b4b05b8725f0fcd7385c0e9dfeef83cda945fc9d496: Status 404 returned error can't find the container with id fb469cbaa7be22931be73b4b05b8725f0fcd7385c0e9dfeef83cda945fc9d496 Jul 26 17:50:31.834728 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:31.834617 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:31.841315 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:31.841291 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gfwst" event={"ID":"67f55fe3-f055-4bd0-8728-bcb5fa86db98","Type":"ContainerStarted","Data":"fb469cbaa7be22931be73b4b05b8725f0fcd7385c0e9dfeef83cda945fc9d496"} Jul 26 17:50:31.931167 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:31.931132 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:32.031010 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:32.030971 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:32.132786 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:32.132660 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:36.382468 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:36.382427 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:36.484135 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:36.484095 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:36.862645 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:36.862612 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gfwst" event={"ID":"67f55fe3-f055-4bd0-8728-bcb5fa86db98","Type":"ContainerStarted","Data":"1958f834f7bc98b1d5b9351593a2e37ac3fadf9333ff588c993b4cfab3b010cf"} Jul 26 17:50:37.219924 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:37.219819 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:37.285138 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:37.285102 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:37.377638 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:37.377552 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:37.487442 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:37.487311 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:37.862080 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:37.861973 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:37.920460 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:37.920419 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:38.066368 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:38.066309 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:38.267679 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:38.267637 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:38.319663 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:38.319616 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:38.875083 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:38.875046 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:39.077071 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:39.077036 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:39.226542 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:39.226510 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:39.853549 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:39.853494 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:39.953001 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:39.952975 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:40.496639 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:40.496603 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:40.597930 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:40.597870 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:40.880791 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:40.880714 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gfwst" event={"ID":"67f55fe3-f055-4bd0-8728-bcb5fa86db98","Type":"ContainerStarted","Data":"04a792c65584aef1005b2c1cd14c661b6fdeea7e7dd6d2638e025d358d230563"} Jul 26 17:50:41.167814 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:41.167730 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:41.228625 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:41.228582 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:41.874898 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:41.874839 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:41.952822 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:41.952786 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:43.380974 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:43.380939 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:43.481988 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:43.481957 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:44.896260 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:44.896227 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gfwst" event={"ID":"67f55fe3-f055-4bd0-8728-bcb5fa86db98","Type":"ContainerStarted","Data":"4beb378511f16bf6592d3d4f69fffad7792f1befaec2388d71f114f57f0b2e9b"} Jul 26 17:50:44.915591 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:44.915517 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gfwst" podStartSLOduration=17.890500887 podStartE2EDuration="29.915501396s" totalImagesPullingTime="12.025000509s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:50:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:50:44.914366772 +0000 UTC m=+98.867022459" watchObservedRunningTime="2026-07-26 17:50:44.915501396 +0000 UTC m=+98.868157083" Jul 26 17:50:45.452784 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:45.452745 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:45.555832 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:45.555796 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:48.934094 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:48.934063 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:49.035974 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:49.035938 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:49.308269 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:49.308233 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:52.804847 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:52.804815 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2sjzw" Jul 26 17:50:53.107143 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:53.107059 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:53.161460 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:53.161425 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:57.302814 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:57.302775 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:57.366591 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:57.366554 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:58.368469 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:58.368437 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:50:58.426186 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:50:58.426150 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:51:06.507276 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:51:06.507239 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:51:09.105263 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:51:09.105227 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:51:09.205960 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:51:09.205929 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:51:10.906278 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:51:10.906233 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:51:11.510288 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:51:11.510252 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:51:11.564715 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:51:11.564684 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:51:13.336237 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:51:13.336192 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:51:13.406700 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:51:13.406640 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:51:13.507828 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:51:13.507794 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:51:28.376253 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:51:28.376205 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:51:28.433846 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:51:28.433809 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:51:44.669050 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:51:44.668950 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:51:44.869361 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:51:44.869321 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:51:45.911369 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:51:45.911331 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:51:45.968801 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:51:45.968768 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:51:54.517903 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:51:54.517858 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:51:54.663573 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:51:54.663540 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:51:54.722135 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:51:54.722103 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:51:55.267921 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:51:55.267885 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:51:57.990491 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:51:57.990439 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:52:05.889916 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:52:05.889872 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:52:05.945785 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:52:05.945757 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:52:05.999386 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:52:05.999357 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:52:31.941742 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:52:31.941699 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:52:59.888173 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:52:59.888139 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:52:59.955150 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:52:59.955097 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:53:04.245879 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:53:04.245840 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:53:04.303069 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:53:04.303039 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:53:11.830847 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:53:11.830808 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:53:11.883312 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:53:11.883273 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:53:15.129656 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:53:15.129618 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:53:15.183284 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:53:15.183242 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:53:15.244272 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:53:15.244239 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:53:28.392059 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:53:28.391965 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:53:28.486749 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:53:28.486713 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:06.363647 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:06.363614 2559 kubelet.go:1648] "Image garbage collection succeeded" Jul 26 17:54:07.491713 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:07.491657 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:08.923868 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:08.923831 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:29.131816 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:29.131777 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:29.942258 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:29.942224 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:32.838417 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:32.838382 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:32.938846 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:32.938815 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:33.742714 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:33.742652 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:33.882506 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:33.882472 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:34.980054 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:34.980020 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:35.078907 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:35.078877 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:37.168218 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:37.168183 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:37.369516 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:37.369469 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:39.525061 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:39.525027 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:39.583891 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:39.583862 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:45.370876 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:45.370839 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:45.548931 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:45.548881 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:45.614666 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:45.614620 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:47.784151 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:47.784115 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4ndxz"] Jul 26 17:54:47.787117 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:47.787101 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4ndxz" Jul 26 17:54:47.789324 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:47.789305 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 26 17:54:47.789495 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:47.789475 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 26 17:54:47.790477 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:47.790459 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-cx5qq\"" Jul 26 17:54:47.795395 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:47.795374 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4ndxz"] Jul 26 17:54:47.956333 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:47.956299 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30b68744-d39f-40e3-8856-26894de4f8a3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-4ndxz\" (UID: \"30b68744-d39f-40e3-8856-26894de4f8a3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4ndxz" Jul 26 17:54:47.956514 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:47.956348 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j2nz\" (UniqueName: \"kubernetes.io/projected/30b68744-d39f-40e3-8856-26894de4f8a3-kube-api-access-7j2nz\") pod \"openshift-lws-operator-fd8ccff4c-4ndxz\" (UID: \"30b68744-d39f-40e3-8856-26894de4f8a3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4ndxz" Jul 26 17:54:48.057739 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:48.057609 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j2nz\" (UniqueName: \"kubernetes.io/projected/30b68744-d39f-40e3-8856-26894de4f8a3-kube-api-access-7j2nz\") pod \"openshift-lws-operator-fd8ccff4c-4ndxz\" (UID: \"30b68744-d39f-40e3-8856-26894de4f8a3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4ndxz" Jul 26 17:54:48.057739 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:48.057729 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30b68744-d39f-40e3-8856-26894de4f8a3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-4ndxz\" (UID: \"30b68744-d39f-40e3-8856-26894de4f8a3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4ndxz" Jul 26 17:54:48.058114 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:48.058092 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30b68744-d39f-40e3-8856-26894de4f8a3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-4ndxz\" (UID: \"30b68744-d39f-40e3-8856-26894de4f8a3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4ndxz" Jul 26 17:54:48.065960 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:48.065933 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j2nz\" (UniqueName: \"kubernetes.io/projected/30b68744-d39f-40e3-8856-26894de4f8a3-kube-api-access-7j2nz\") pod \"openshift-lws-operator-fd8ccff4c-4ndxz\" (UID: \"30b68744-d39f-40e3-8856-26894de4f8a3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4ndxz" Jul 26 17:54:48.096613 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:48.096564 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4ndxz" Jul 26 17:54:48.222835 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:48.222796 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4ndxz"] Jul 26 17:54:48.225617 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:54:48.225584 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30b68744_d39f_40e3_8856_26894de4f8a3.slice/crio-1f6bd89ddb95bfe2cfc7526f884c07f633aefb6e550fb9cd8a31e112df836784 WatchSource:0}: Error finding container 1f6bd89ddb95bfe2cfc7526f884c07f633aefb6e550fb9cd8a31e112df836784: Status 404 returned error can't find the container with id 1f6bd89ddb95bfe2cfc7526f884c07f633aefb6e550fb9cd8a31e112df836784 Jul 26 17:54:48.227149 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:48.227130 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 26 17:54:48.711701 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:48.711653 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4ndxz" event={"ID":"30b68744-d39f-40e3-8856-26894de4f8a3","Type":"ContainerStarted","Data":"1f6bd89ddb95bfe2cfc7526f884c07f633aefb6e550fb9cd8a31e112df836784"} Jul 26 17:54:49.445541 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:49.441955 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:49.495690 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:49.495635 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:51.723927 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:51.723890 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4ndxz" event={"ID":"30b68744-d39f-40e3-8856-26894de4f8a3","Type":"ContainerStarted","Data":"a93158b18d5d3e178f6a0dda898e72a9b86804dd097a7f32c043305c419325ba"} Jul 26 17:54:51.739788 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:51.739734 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4ndxz" podStartSLOduration=2.254389571 podStartE2EDuration="4.73971906s" totalImagesPullingTime="2.485329489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:54:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:54:51.738845697 +0000 UTC m=+345.691501384" watchObservedRunningTime="2026-07-26 17:54:51.73971906 +0000 UTC m=+345.692374748" Jul 26 17:54:52.433382 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:52.433335 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:52.540427 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:52.540383 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:54:54.481542 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:54:54.481511 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:07.145644 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:07.145558 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:07.240383 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:07.240340 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:07.743214 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:07.743178 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:07.960139 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:07.960108 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:08.055754 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.055658 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:08.141448 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.141413 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd"] Jul 26 17:55:08.146364 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.146343 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" Jul 26 17:55:08.150751 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:55:08.150717 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"opendatahub-operator-controller-manager-7b79b7df6c-rjcmd\" is forbidden: User \"system:node:ip-10-0-128-209.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-209.ec2.internal' and this object" podUID="2cec0f48-bf3f-4a86-b72d-602edc087625" pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" Jul 26 17:55:08.150751 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:55:08.150718 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-dockercfg-ml4x7\" is forbidden: User \"system:node:ip-10-0-128-209.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-209.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ml4x7\"" type="*v1.Secret" Jul 26 17:55:08.150941 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:55:08.150760 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-128-209.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-209.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 26 17:55:08.151378 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.151362 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 26 17:55:08.151491 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.151472 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 26 17:55:08.151601 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.151583 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 26 17:55:08.179795 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.179767 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd"] Jul 26 17:55:08.200467 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.200433 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2cec0f48-bf3f-4a86-b72d-602edc087625-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-rjcmd\" (UID: \"2cec0f48-bf3f-4a86-b72d-602edc087625\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" Jul 26 17:55:08.200467 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.200474 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2cec0f48-bf3f-4a86-b72d-602edc087625-manifests\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-rjcmd\" (UID: \"2cec0f48-bf3f-4a86-b72d-602edc087625\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" Jul 26 17:55:08.200703 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.200504 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2cec0f48-bf3f-4a86-b72d-602edc087625-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-rjcmd\" (UID: \"2cec0f48-bf3f-4a86-b72d-602edc087625\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" Jul 26 17:55:08.200703 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.200571 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2cec0f48-bf3f-4a86-b72d-602edc087625-tmp\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-rjcmd\" (UID: \"2cec0f48-bf3f-4a86-b72d-602edc087625\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" Jul 26 17:55:08.200703 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.200609 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdcwn\" (UniqueName: \"kubernetes.io/projected/2cec0f48-bf3f-4a86-b72d-602edc087625-kube-api-access-pdcwn\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-rjcmd\" (UID: \"2cec0f48-bf3f-4a86-b72d-602edc087625\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" Jul 26 17:55:08.301202 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.301158 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2cec0f48-bf3f-4a86-b72d-602edc087625-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-rjcmd\" (UID: \"2cec0f48-bf3f-4a86-b72d-602edc087625\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" Jul 26 17:55:08.301369 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.301214 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2cec0f48-bf3f-4a86-b72d-602edc087625-manifests\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-rjcmd\" (UID: \"2cec0f48-bf3f-4a86-b72d-602edc087625\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" Jul 26 17:55:08.301369 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.301287 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2cec0f48-bf3f-4a86-b72d-602edc087625-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-rjcmd\" (UID: \"2cec0f48-bf3f-4a86-b72d-602edc087625\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" Jul 26 17:55:08.301369 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.301320 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2cec0f48-bf3f-4a86-b72d-602edc087625-tmp\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-rjcmd\" (UID: \"2cec0f48-bf3f-4a86-b72d-602edc087625\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" Jul 26 17:55:08.301369 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.301340 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdcwn\" (UniqueName: \"kubernetes.io/projected/2cec0f48-bf3f-4a86-b72d-602edc087625-kube-api-access-pdcwn\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-rjcmd\" (UID: \"2cec0f48-bf3f-4a86-b72d-602edc087625\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" Jul 26 17:55:08.301741 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.301706 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2cec0f48-bf3f-4a86-b72d-602edc087625-manifests\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-rjcmd\" (UID: \"2cec0f48-bf3f-4a86-b72d-602edc087625\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" Jul 26 17:55:08.303486 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.303467 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2cec0f48-bf3f-4a86-b72d-602edc087625-tmp\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-rjcmd\" (UID: \"2cec0f48-bf3f-4a86-b72d-602edc087625\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" Jul 26 17:55:08.303611 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.303594 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2cec0f48-bf3f-4a86-b72d-602edc087625-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-rjcmd\" (UID: \"2cec0f48-bf3f-4a86-b72d-602edc087625\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" Jul 26 17:55:08.303709 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.303690 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2cec0f48-bf3f-4a86-b72d-602edc087625-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-rjcmd\" (UID: \"2cec0f48-bf3f-4a86-b72d-602edc087625\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" Jul 26 17:55:08.309592 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.309515 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:08.410842 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.410808 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:08.517426 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.517391 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:08.599409 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.599317 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:08.663693 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.663633 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:08.764104 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.764074 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:08.864010 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:08.863923 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:09.111306 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:09.111272 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ml4x7\"" Jul 26 17:55:09.308244 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:55:09.308194 2559 projected.go:291] Couldn't get configMap opendatahub/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Jul 26 17:55:09.308244 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:55:09.308256 2559 projected.go:196] Error preparing data for projected volume kube-api-access-pdcwn for pod opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd: failed to sync configmap cache: timed out waiting for the condition Jul 26 17:55:09.308705 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:55:09.308333 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2cec0f48-bf3f-4a86-b72d-602edc087625-kube-api-access-pdcwn podName:2cec0f48-bf3f-4a86-b72d-602edc087625 nodeName:}" failed. No retries permitted until 2026-07-26 17:55:09.808317977 +0000 UTC m=+363.760973643 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pdcwn" (UniqueName: "kubernetes.io/projected/2cec0f48-bf3f-4a86-b72d-602edc087625-kube-api-access-pdcwn") pod "opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" (UID: "2cec0f48-bf3f-4a86-b72d-602edc087625") : failed to sync configmap cache: timed out waiting for the condition Jul 26 17:55:09.383161 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:09.383133 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 26 17:55:09.813162 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:09.813118 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdcwn\" (UniqueName: \"kubernetes.io/projected/2cec0f48-bf3f-4a86-b72d-602edc087625-kube-api-access-pdcwn\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-rjcmd\" (UID: \"2cec0f48-bf3f-4a86-b72d-602edc087625\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" Jul 26 17:55:09.815811 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:09.815777 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdcwn\" (UniqueName: \"kubernetes.io/projected/2cec0f48-bf3f-4a86-b72d-602edc087625-kube-api-access-pdcwn\") pod \"opendatahub-operator-controller-manager-7b79b7df6c-rjcmd\" (UID: \"2cec0f48-bf3f-4a86-b72d-602edc087625\") " pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" Jul 26 17:55:09.956639 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:09.956597 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" Jul 26 17:55:10.085729 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:10.085705 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd"] Jul 26 17:55:10.087360 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:55:10.087337 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cec0f48_bf3f_4a86_b72d_602edc087625.slice/crio-057b49338fcf78d2e3aae5296b13f9bdad0438fcdc723f27e9b608b94094372b WatchSource:0}: Error finding container 057b49338fcf78d2e3aae5296b13f9bdad0438fcdc723f27e9b608b94094372b: Status 404 returned error can't find the container with id 057b49338fcf78d2e3aae5296b13f9bdad0438fcdc723f27e9b608b94094372b Jul 26 17:55:10.789196 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:10.789151 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" event={"ID":"2cec0f48-bf3f-4a86-b72d-602edc087625","Type":"ContainerStarted","Data":"057b49338fcf78d2e3aae5296b13f9bdad0438fcdc723f27e9b608b94094372b"} Jul 26 17:55:13.802009 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:13.801974 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" event={"ID":"2cec0f48-bf3f-4a86-b72d-602edc087625","Type":"ContainerStarted","Data":"21cfe17fa96aafcce4fc6a877dfe7547a415646e041e6301641d1d978bed5313"} Jul 26 17:55:14.806636 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:14.806598 2559 generic.go:356] "Generic (PLEG): container finished" podID="2cec0f48-bf3f-4a86-b72d-602edc087625" containerID="21cfe17fa96aafcce4fc6a877dfe7547a415646e041e6301641d1d978bed5313" exitCode=0 Jul 26 17:55:14.807020 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:14.806654 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" event={"ID":"2cec0f48-bf3f-4a86-b72d-602edc087625","Type":"ContainerDied","Data":"21cfe17fa96aafcce4fc6a877dfe7547a415646e041e6301641d1d978bed5313"} Jul 26 17:55:15.811638 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:15.811606 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" event={"ID":"2cec0f48-bf3f-4a86-b72d-602edc087625","Type":"ContainerStarted","Data":"7941f2f7ac272b144037cbe9a4df55d49058c0eea80ab4a7e1fc8f2e85a9aba2"} Jul 26 17:55:15.812035 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:15.811709 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" Jul 26 17:55:15.832724 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:15.832651 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" podStartSLOduration=3.735132654 podStartE2EDuration="7.832634721s" totalImagesPullingTime="3.066870918s" totalInitContainerRuntime="1.030631149s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:55:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:55:15.831048374 +0000 UTC m=+369.783704073" watchObservedRunningTime="2026-07-26 17:55:15.832634721 +0000 UTC m=+369.785290408" Jul 26 17:55:19.293332 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:19.293293 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:19.393720 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:19.393691 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:22.519772 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:22.519725 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:22.681384 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:22.681347 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:26.817436 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:26.817406 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b79b7df6c-rjcmd" Jul 26 17:55:28.341830 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:28.341786 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:28.468339 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:28.468304 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:28.888467 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:28.888428 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:29.052150 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.052114 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6855b89bd5-2586g"] Jul 26 17:55:29.055272 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.055252 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6855b89bd5-2586g" Jul 26 17:55:29.058607 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.058582 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 26 17:55:29.058607 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.058589 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 26 17:55:29.058831 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.058687 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 26 17:55:29.058831 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.058700 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 26 17:55:29.059641 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.059622 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-r79c6\"" Jul 26 17:55:29.066231 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.066201 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6855b89bd5-2586g"] Jul 26 17:55:29.159311 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.159225 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31bcded3-9797-486d-b175-9ef1fc6d0b1a-tmp\") pod \"kube-auth-proxy-6855b89bd5-2586g\" (UID: \"31bcded3-9797-486d-b175-9ef1fc6d0b1a\") " pod="openshift-ingress/kube-auth-proxy-6855b89bd5-2586g" Jul 26 17:55:29.159311 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.159282 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31bcded3-9797-486d-b175-9ef1fc6d0b1a-tls-certs\") pod \"kube-auth-proxy-6855b89bd5-2586g\" (UID: \"31bcded3-9797-486d-b175-9ef1fc6d0b1a\") " pod="openshift-ingress/kube-auth-proxy-6855b89bd5-2586g" Jul 26 17:55:29.159489 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.159358 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cd6x\" (UniqueName: \"kubernetes.io/projected/31bcded3-9797-486d-b175-9ef1fc6d0b1a-kube-api-access-8cd6x\") pod \"kube-auth-proxy-6855b89bd5-2586g\" (UID: \"31bcded3-9797-486d-b175-9ef1fc6d0b1a\") " pod="openshift-ingress/kube-auth-proxy-6855b89bd5-2586g" Jul 26 17:55:29.218445 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.218415 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:29.260289 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.260252 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cd6x\" (UniqueName: \"kubernetes.io/projected/31bcded3-9797-486d-b175-9ef1fc6d0b1a-kube-api-access-8cd6x\") pod \"kube-auth-proxy-6855b89bd5-2586g\" (UID: \"31bcded3-9797-486d-b175-9ef1fc6d0b1a\") " pod="openshift-ingress/kube-auth-proxy-6855b89bd5-2586g" Jul 26 17:55:29.260448 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.260337 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31bcded3-9797-486d-b175-9ef1fc6d0b1a-tmp\") pod \"kube-auth-proxy-6855b89bd5-2586g\" (UID: \"31bcded3-9797-486d-b175-9ef1fc6d0b1a\") " pod="openshift-ingress/kube-auth-proxy-6855b89bd5-2586g" Jul 26 17:55:29.260448 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.260405 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31bcded3-9797-486d-b175-9ef1fc6d0b1a-tls-certs\") pod \"kube-auth-proxy-6855b89bd5-2586g\" (UID: \"31bcded3-9797-486d-b175-9ef1fc6d0b1a\") " pod="openshift-ingress/kube-auth-proxy-6855b89bd5-2586g" Jul 26 17:55:29.262517 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.262494 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31bcded3-9797-486d-b175-9ef1fc6d0b1a-tmp\") pod \"kube-auth-proxy-6855b89bd5-2586g\" (UID: \"31bcded3-9797-486d-b175-9ef1fc6d0b1a\") " pod="openshift-ingress/kube-auth-proxy-6855b89bd5-2586g" Jul 26 17:55:29.262923 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.262902 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31bcded3-9797-486d-b175-9ef1fc6d0b1a-tls-certs\") pod \"kube-auth-proxy-6855b89bd5-2586g\" (UID: \"31bcded3-9797-486d-b175-9ef1fc6d0b1a\") " pod="openshift-ingress/kube-auth-proxy-6855b89bd5-2586g" Jul 26 17:55:29.272597 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.272568 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cd6x\" (UniqueName: \"kubernetes.io/projected/31bcded3-9797-486d-b175-9ef1fc6d0b1a-kube-api-access-8cd6x\") pod \"kube-auth-proxy-6855b89bd5-2586g\" (UID: \"31bcded3-9797-486d-b175-9ef1fc6d0b1a\") " pod="openshift-ingress/kube-auth-proxy-6855b89bd5-2586g" Jul 26 17:55:29.318957 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.318918 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:29.366755 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.366718 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6855b89bd5-2586g" Jul 26 17:55:29.426059 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.425967 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:29.504657 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.504622 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6855b89bd5-2586g"] Jul 26 17:55:29.507520 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:55:29.507482 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31bcded3_9797_486d_b175_9ef1fc6d0b1a.slice/crio-ff5b8d6190271ab03fe9214a64e7c2f3ffa1938c75d2a07d3fbf312bbae5516a WatchSource:0}: Error finding container ff5b8d6190271ab03fe9214a64e7c2f3ffa1938c75d2a07d3fbf312bbae5516a: Status 404 returned error can't find the container with id ff5b8d6190271ab03fe9214a64e7c2f3ffa1938c75d2a07d3fbf312bbae5516a Jul 26 17:55:29.519145 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.519117 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:29.606450 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.606414 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:29.823886 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.823853 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:29.865017 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:29.864982 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6855b89bd5-2586g" event={"ID":"31bcded3-9797-486d-b175-9ef1fc6d0b1a","Type":"ContainerStarted","Data":"ff5b8d6190271ab03fe9214a64e7c2f3ffa1938c75d2a07d3fbf312bbae5516a"} Jul 26 17:55:30.225268 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:30.225231 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:30.301025 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:30.300987 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jjvgl"] Jul 26 17:55:30.304525 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:30.304493 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-jjvgl" Jul 26 17:55:30.308374 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:30.308283 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 26 17:55:30.310373 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:30.310337 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:30.320746 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:30.320611 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jjvgl"] Jul 26 17:55:30.369333 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:30.369295 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23-tls-certs\") pod \"model-serving-api-54d89bd5fc-jjvgl\" (UID: \"63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23\") " pod="opendatahub/model-serving-api-54d89bd5fc-jjvgl" Jul 26 17:55:30.369746 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:30.369340 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z8tj\" (UniqueName: \"kubernetes.io/projected/63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23-kube-api-access-2z8tj\") pod \"model-serving-api-54d89bd5fc-jjvgl\" (UID: \"63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23\") " pod="opendatahub/model-serving-api-54d89bd5fc-jjvgl" Jul 26 17:55:30.470551 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:30.470519 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23-tls-certs\") pod \"model-serving-api-54d89bd5fc-jjvgl\" (UID: \"63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23\") " pod="opendatahub/model-serving-api-54d89bd5fc-jjvgl" Jul 26 17:55:30.470770 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:30.470562 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z8tj\" (UniqueName: \"kubernetes.io/projected/63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23-kube-api-access-2z8tj\") pod \"model-serving-api-54d89bd5fc-jjvgl\" (UID: \"63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23\") " pod="opendatahub/model-serving-api-54d89bd5fc-jjvgl" Jul 26 17:55:30.470770 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:55:30.470702 2559 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 26 17:55:30.470874 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:55:30.470776 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23-tls-certs podName:63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23 nodeName:}" failed. No retries permitted until 2026-07-26 17:55:30.970754376 +0000 UTC m=+384.923410052 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23-tls-certs") pod "model-serving-api-54d89bd5fc-jjvgl" (UID: "63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23") : secret "model-serving-api-tls" not found Jul 26 17:55:30.493659 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:30.493590 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z8tj\" (UniqueName: \"kubernetes.io/projected/63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23-kube-api-access-2z8tj\") pod \"model-serving-api-54d89bd5fc-jjvgl\" (UID: \"63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23\") " pod="opendatahub/model-serving-api-54d89bd5fc-jjvgl" Jul 26 17:55:30.769766 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:30.769665 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:30.976108 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:30.976069 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23-tls-certs\") pod \"model-serving-api-54d89bd5fc-jjvgl\" (UID: \"63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23\") " pod="opendatahub/model-serving-api-54d89bd5fc-jjvgl" Jul 26 17:55:30.978920 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:30.978890 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23-tls-certs\") pod \"model-serving-api-54d89bd5fc-jjvgl\" (UID: \"63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23\") " pod="opendatahub/model-serving-api-54d89bd5fc-jjvgl" Jul 26 17:55:31.217480 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:31.217449 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-jjvgl" Jul 26 17:55:31.352000 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:31.351957 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jjvgl"] Jul 26 17:55:31.352531 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:31.352486 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:31.355169 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:55:31.355137 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63332e8b_dfa5_4aeb_bcb9_b7166a0c7a23.slice/crio-60d02c3e2bb7fdebeaed17ef60867c40d55941ffd6a6cadca2d89660bf9f6875 WatchSource:0}: Error finding container 60d02c3e2bb7fdebeaed17ef60867c40d55941ffd6a6cadca2d89660bf9f6875: Status 404 returned error can't find the container with id 60d02c3e2bb7fdebeaed17ef60867c40d55941ffd6a6cadca2d89660bf9f6875 Jul 26 17:55:31.875470 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:31.875429 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-jjvgl" event={"ID":"63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23","Type":"ContainerStarted","Data":"60d02c3e2bb7fdebeaed17ef60867c40d55941ffd6a6cadca2d89660bf9f6875"} Jul 26 17:55:32.882462 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:32.882421 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6855b89bd5-2586g" event={"ID":"31bcded3-9797-486d-b175-9ef1fc6d0b1a","Type":"ContainerStarted","Data":"1e1cafe8763d123270711a0cb4126c1206bf1cd345d55e36346f3805ff89d350"} Jul 26 17:55:32.900832 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:32.900776 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6855b89bd5-2586g" podStartSLOduration=1.329332524 podStartE2EDuration="3.900758117s" totalImagesPullingTime="2.571425593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:55:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:55:32.899487627 +0000 UTC m=+386.852143316" watchObservedRunningTime="2026-07-26 17:55:32.900758117 +0000 UTC m=+386.853413812" Jul 26 17:55:33.889844 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:33.889803 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-jjvgl" event={"ID":"63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23","Type":"ContainerStarted","Data":"eb5eb6cfc8ce113ee4ae19e3dc2ffc79b3491f02f1fc068c5f9816527ffd18b0"} Jul 26 17:55:33.890285 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:33.889980 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-jjvgl" Jul 26 17:55:33.914589 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:33.914546 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-jjvgl" podStartSLOduration=2.185358424 podStartE2EDuration="3.914532061s" totalImagesPullingTime="1.729173637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:55:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:55:33.914054148 +0000 UTC m=+387.866709836" watchObservedRunningTime="2026-07-26 17:55:33.914532061 +0000 UTC m=+387.867187749" Jul 26 17:55:35.933562 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:35.933525 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:36.032062 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.032031 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:36.302722 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.302686 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd"] Jul 26 17:55:36.305928 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.305910 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.308583 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.308563 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 26 17:55:36.309202 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.309184 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-pg7gm\"" Jul 26 17:55:36.309340 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.309325 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 26 17:55:36.309611 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.309596 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 26 17:55:36.309611 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.309605 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 26 17:55:36.318710 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.318688 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd"] Jul 26 17:55:36.423757 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.423723 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhxvk\" (UniqueName: \"kubernetes.io/projected/f3875b31-ffe3-48dd-b240-0e799813a0ef-kube-api-access-lhxvk\") pod \"istiod-openshift-gateway-5677ccb49b-xmcxd\" (UID: \"f3875b31-ffe3-48dd-b240-0e799813a0ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.423956 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.423775 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f3875b31-ffe3-48dd-b240-0e799813a0ef-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-xmcxd\" (UID: \"f3875b31-ffe3-48dd-b240-0e799813a0ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.423956 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.423843 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f3875b31-ffe3-48dd-b240-0e799813a0ef-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-xmcxd\" (UID: \"f3875b31-ffe3-48dd-b240-0e799813a0ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.423956 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.423923 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3875b31-ffe3-48dd-b240-0e799813a0ef-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-xmcxd\" (UID: \"f3875b31-ffe3-48dd-b240-0e799813a0ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.424080 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.423967 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f3875b31-ffe3-48dd-b240-0e799813a0ef-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-xmcxd\" (UID: \"f3875b31-ffe3-48dd-b240-0e799813a0ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.424080 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.423995 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3875b31-ffe3-48dd-b240-0e799813a0ef-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-xmcxd\" (UID: \"f3875b31-ffe3-48dd-b240-0e799813a0ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.424080 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.424014 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f3875b31-ffe3-48dd-b240-0e799813a0ef-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-xmcxd\" (UID: \"f3875b31-ffe3-48dd-b240-0e799813a0ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.525008 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.524977 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhxvk\" (UniqueName: \"kubernetes.io/projected/f3875b31-ffe3-48dd-b240-0e799813a0ef-kube-api-access-lhxvk\") pod \"istiod-openshift-gateway-5677ccb49b-xmcxd\" (UID: \"f3875b31-ffe3-48dd-b240-0e799813a0ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.525008 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.525015 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f3875b31-ffe3-48dd-b240-0e799813a0ef-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-xmcxd\" (UID: \"f3875b31-ffe3-48dd-b240-0e799813a0ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.525263 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.525045 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f3875b31-ffe3-48dd-b240-0e799813a0ef-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-xmcxd\" (UID: \"f3875b31-ffe3-48dd-b240-0e799813a0ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.525263 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.525101 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3875b31-ffe3-48dd-b240-0e799813a0ef-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-xmcxd\" (UID: \"f3875b31-ffe3-48dd-b240-0e799813a0ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.525263 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.525132 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f3875b31-ffe3-48dd-b240-0e799813a0ef-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-xmcxd\" (UID: \"f3875b31-ffe3-48dd-b240-0e799813a0ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.525263 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.525158 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3875b31-ffe3-48dd-b240-0e799813a0ef-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-xmcxd\" (UID: \"f3875b31-ffe3-48dd-b240-0e799813a0ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.525263 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.525187 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f3875b31-ffe3-48dd-b240-0e799813a0ef-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-xmcxd\" (UID: \"f3875b31-ffe3-48dd-b240-0e799813a0ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.525759 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.525733 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f3875b31-ffe3-48dd-b240-0e799813a0ef-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-xmcxd\" (UID: \"f3875b31-ffe3-48dd-b240-0e799813a0ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.527586 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.527560 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3875b31-ffe3-48dd-b240-0e799813a0ef-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-xmcxd\" (UID: \"f3875b31-ffe3-48dd-b240-0e799813a0ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.527768 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.527747 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f3875b31-ffe3-48dd-b240-0e799813a0ef-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-xmcxd\" (UID: \"f3875b31-ffe3-48dd-b240-0e799813a0ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.527823 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.527774 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f3875b31-ffe3-48dd-b240-0e799813a0ef-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-xmcxd\" (UID: \"f3875b31-ffe3-48dd-b240-0e799813a0ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.527823 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.527791 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f3875b31-ffe3-48dd-b240-0e799813a0ef-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-xmcxd\" (UID: \"f3875b31-ffe3-48dd-b240-0e799813a0ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.537011 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.536991 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhxvk\" (UniqueName: \"kubernetes.io/projected/f3875b31-ffe3-48dd-b240-0e799813a0ef-kube-api-access-lhxvk\") pod \"istiod-openshift-gateway-5677ccb49b-xmcxd\" (UID: \"f3875b31-ffe3-48dd-b240-0e799813a0ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.539659 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.539641 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3875b31-ffe3-48dd-b240-0e799813a0ef-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-xmcxd\" (UID: \"f3875b31-ffe3-48dd-b240-0e799813a0ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.614590 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.614497 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:36.758974 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.758932 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd"] Jul 26 17:55:36.761318 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.761288 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:36.762191 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:55:36.762165 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3875b31_ffe3_48dd_b240_0e799813a0ef.slice/crio-d3f5f4492fff38d3cca78249fe14d6285a6901c28939ac3c9e4e76713d8ca215 WatchSource:0}: Error finding container d3f5f4492fff38d3cca78249fe14d6285a6901c28939ac3c9e4e76713d8ca215: Status 404 returned error can't find the container with id d3f5f4492fff38d3cca78249fe14d6285a6901c28939ac3c9e4e76713d8ca215 Jul 26 17:55:36.908946 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.908869 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" event={"ID":"f3875b31-ffe3-48dd-b240-0e799813a0ef","Type":"ContainerStarted","Data":"d3f5f4492fff38d3cca78249fe14d6285a6901c28939ac3c9e4e76713d8ca215"} Jul 26 17:55:36.951188 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:36.951146 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:37.856217 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:37.856170 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:39.178305 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:39.178268 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:39.309861 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:39.309824 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:39.502627 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:39.502591 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:39.604314 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:39.604278 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:39.711270 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:39.711233 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:39.945274 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:39.945183 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:55:40.284510 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:40.284473 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 26 17:55:40.284770 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:40.284557 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 26 17:55:40.928814 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:40.928775 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" event={"ID":"f3875b31-ffe3-48dd-b240-0e799813a0ef","Type":"ContainerStarted","Data":"cff2ec301666109840c077e19fac08f7395f472239df2c89475cdf8523cbec8d"} Jul 26 17:55:40.929703 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:40.929661 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:40.969620 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:40.969563 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" podStartSLOduration=1.4494247470000001 podStartE2EDuration="4.969546875s" totalImagesPullingTime="3.520122128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:55:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:55:40.967999086 +0000 UTC m=+394.920654774" watchObservedRunningTime="2026-07-26 17:55:40.969546875 +0000 UTC m=+394.922202562" Jul 26 17:55:41.935140 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:41.935114 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-xmcxd" Jul 26 17:55:44.897830 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:55:44.897802 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-jjvgl" Jul 26 17:56:18.899941 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:18.899910 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:56:29.765322 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:29.765243 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:56:29.856319 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:29.856286 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:56:29.860356 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:29.860331 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-shw47"] Jul 26 17:56:29.863463 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:29.863442 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-shw47" Jul 26 17:56:29.866430 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:29.866409 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 26 17:56:29.866556 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:29.866414 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-bvchv\"" Jul 26 17:56:29.867166 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:29.867151 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 26 17:56:29.875611 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:29.875586 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-shw47"] Jul 26 17:56:29.926359 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:29.926316 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:56:29.933384 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:29.933364 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zqxh\" (UniqueName: \"kubernetes.io/projected/bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1-kube-api-access-9zqxh\") pod \"kuadrant-operator-catalog-shw47\" (UID: \"bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1\") " pod="kuadrant-system/kuadrant-operator-catalog-shw47" Jul 26 17:56:30.034219 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:30.034125 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zqxh\" (UniqueName: \"kubernetes.io/projected/bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1-kube-api-access-9zqxh\") pod \"kuadrant-operator-catalog-shw47\" (UID: \"bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1\") " pod="kuadrant-system/kuadrant-operator-catalog-shw47" Jul 26 17:56:30.048975 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:30.048941 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zqxh\" (UniqueName: \"kubernetes.io/projected/bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1-kube-api-access-9zqxh\") pod \"kuadrant-operator-catalog-shw47\" (UID: \"bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1\") " pod="kuadrant-system/kuadrant-operator-catalog-shw47" Jul 26 17:56:30.173496 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:30.173454 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-shw47" Jul 26 17:56:30.218948 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:30.218915 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-shw47"] Jul 26 17:56:30.301272 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:30.301194 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-shw47"] Jul 26 17:56:30.303711 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:56:30.303686 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfb4a8c8_8dbd_4004_a2b5_7d40bb5e1ef1.slice/crio-edd7a15b4aa52e0a22ab16db9b25761511e547cf6673b169ff74708e06e8dd0d WatchSource:0}: Error finding container edd7a15b4aa52e0a22ab16db9b25761511e547cf6673b169ff74708e06e8dd0d: Status 404 returned error can't find the container with id edd7a15b4aa52e0a22ab16db9b25761511e547cf6673b169ff74708e06e8dd0d Jul 26 17:56:31.127411 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:31.127379 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-shw47" event={"ID":"bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1","Type":"ContainerStarted","Data":"edd7a15b4aa52e0a22ab16db9b25761511e547cf6673b169ff74708e06e8dd0d"} Jul 26 17:56:33.138546 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:33.138513 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-shw47" event={"ID":"bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1","Type":"ContainerStarted","Data":"53b876f26a3f01f98d972cbeb3ebc55950dc4131910a4a291f220b49359d1ca5"} Jul 26 17:56:33.139024 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:33.138607 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-shw47" podUID="bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1" containerName="registry-server" containerID="cri-o://53b876f26a3f01f98d972cbeb3ebc55950dc4131910a4a291f220b49359d1ca5" gracePeriod=2 Jul 26 17:56:33.153210 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:33.153171 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-shw47" podStartSLOduration=2.032072897 podStartE2EDuration="4.153155727s" totalImagesPullingTime="2.12108283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:56:33.152304082 +0000 UTC m=+447.104959771" watchObservedRunningTime="2026-07-26 17:56:33.153155727 +0000 UTC m=+447.105811416" Jul 26 17:56:33.380645 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:33.380619 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-shw47" Jul 26 17:56:33.468103 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:33.468071 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zqxh\" (UniqueName: \"kubernetes.io/projected/bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1-kube-api-access-9zqxh\") pod \"bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1\" (UID: \"bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1\") " Jul 26 17:56:33.470309 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:33.470270 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1-kube-api-access-9zqxh" (OuterVolumeSpecName: "kube-api-access-9zqxh") pod "bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1" (UID: "bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1"). InnerVolumeSpecName "kube-api-access-9zqxh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 17:56:33.568943 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:33.568907 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zqxh\" (UniqueName: \"kubernetes.io/projected/bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1-kube-api-access-9zqxh\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 17:56:34.144454 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:34.144423 2559 generic.go:356] "Generic (PLEG): container finished" podID="bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1" containerID="53b876f26a3f01f98d972cbeb3ebc55950dc4131910a4a291f220b49359d1ca5" exitCode=0 Jul 26 17:56:34.144886 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:34.144465 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-shw47" Jul 26 17:56:34.144886 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:34.144513 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-shw47" event={"ID":"bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1","Type":"ContainerDied","Data":"53b876f26a3f01f98d972cbeb3ebc55950dc4131910a4a291f220b49359d1ca5"} Jul 26 17:56:34.144886 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:34.144557 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-shw47" event={"ID":"bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1","Type":"ContainerDied","Data":"edd7a15b4aa52e0a22ab16db9b25761511e547cf6673b169ff74708e06e8dd0d"} Jul 26 17:56:34.144886 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:34.144588 2559 scope.go:117] "RemoveContainer" containerID="53b876f26a3f01f98d972cbeb3ebc55950dc4131910a4a291f220b49359d1ca5" Jul 26 17:56:34.153789 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:34.153772 2559 scope.go:117] "RemoveContainer" containerID="53b876f26a3f01f98d972cbeb3ebc55950dc4131910a4a291f220b49359d1ca5" Jul 26 17:56:34.154086 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:56:34.154066 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53b876f26a3f01f98d972cbeb3ebc55950dc4131910a4a291f220b49359d1ca5\": container with ID starting with 53b876f26a3f01f98d972cbeb3ebc55950dc4131910a4a291f220b49359d1ca5 not found: ID does not exist" containerID="53b876f26a3f01f98d972cbeb3ebc55950dc4131910a4a291f220b49359d1ca5" Jul 26 17:56:34.154137 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:34.154095 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53b876f26a3f01f98d972cbeb3ebc55950dc4131910a4a291f220b49359d1ca5"} err="failed to get container status \"53b876f26a3f01f98d972cbeb3ebc55950dc4131910a4a291f220b49359d1ca5\": rpc error: code = NotFound desc = could not find container \"53b876f26a3f01f98d972cbeb3ebc55950dc4131910a4a291f220b49359d1ca5\": container with ID starting with 53b876f26a3f01f98d972cbeb3ebc55950dc4131910a4a291f220b49359d1ca5 not found: ID does not exist" Jul 26 17:56:34.164474 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:34.164444 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-shw47"] Jul 26 17:56:34.168009 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:34.167988 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-shw47"] Jul 26 17:56:34.437901 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:34.437819 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1" path="/var/lib/kubelet/pods/bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1/volumes" Jul 26 17:56:58.174980 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:58.174945 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:56:58.777313 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:58.777261 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:56:59.816449 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:56:59.816413 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:01.691977 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:01.691942 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-t2h8t"] Jul 26 17:57:01.692370 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:01.692222 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1" containerName="registry-server" Jul 26 17:57:01.692370 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:01.692232 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1" containerName="registry-server" Jul 26 17:57:01.692370 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:01.692298 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfb4a8c8-8dbd-4004-a2b5-7d40bb5e1ef1" containerName="registry-server" Jul 26 17:57:01.695451 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:01.695433 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-t2h8t" Jul 26 17:57:01.698816 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:01.698793 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-ghsqk\"" Jul 26 17:57:01.698949 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:01.698937 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 26 17:57:01.700703 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:01.700665 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 26 17:57:01.709577 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:01.709554 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-t2h8t"] Jul 26 17:57:01.809891 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:01.809854 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvr2k\" (UniqueName: \"kubernetes.io/projected/1d8e314e-f2cb-4018-aa98-cff979e2fba8-kube-api-access-rvr2k\") pod \"authorino-operator-75cc8d5cb7-t2h8t\" (UID: \"1d8e314e-f2cb-4018-aa98-cff979e2fba8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-t2h8t" Jul 26 17:57:01.910623 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:01.910591 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvr2k\" (UniqueName: \"kubernetes.io/projected/1d8e314e-f2cb-4018-aa98-cff979e2fba8-kube-api-access-rvr2k\") pod \"authorino-operator-75cc8d5cb7-t2h8t\" (UID: \"1d8e314e-f2cb-4018-aa98-cff979e2fba8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-t2h8t" Jul 26 17:57:01.920491 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:01.920453 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvr2k\" (UniqueName: \"kubernetes.io/projected/1d8e314e-f2cb-4018-aa98-cff979e2fba8-kube-api-access-rvr2k\") pod \"authorino-operator-75cc8d5cb7-t2h8t\" (UID: \"1d8e314e-f2cb-4018-aa98-cff979e2fba8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-t2h8t" Jul 26 17:57:02.005874 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:02.005835 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-t2h8t" Jul 26 17:57:02.137842 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:02.137817 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-t2h8t"] Jul 26 17:57:02.140310 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:57:02.140282 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d8e314e_f2cb_4018_aa98_cff979e2fba8.slice/crio-e2360affebb5dc453ef020e532ca3b7f5f5b9c4f0bac5f985d0a78d1ca283bcb WatchSource:0}: Error finding container e2360affebb5dc453ef020e532ca3b7f5f5b9c4f0bac5f985d0a78d1ca283bcb: Status 404 returned error can't find the container with id e2360affebb5dc453ef020e532ca3b7f5f5b9c4f0bac5f985d0a78d1ca283bcb Jul 26 17:57:02.257625 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:02.257541 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-t2h8t" event={"ID":"1d8e314e-f2cb-4018-aa98-cff979e2fba8","Type":"ContainerStarted","Data":"e2360affebb5dc453ef020e532ca3b7f5f5b9c4f0bac5f985d0a78d1ca283bcb"} Jul 26 17:57:03.010722 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:03.010653 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:03.569162 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:03.569125 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:05.271332 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:05.271300 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-t2h8t" event={"ID":"1d8e314e-f2cb-4018-aa98-cff979e2fba8","Type":"ContainerStarted","Data":"6a47b28ec50967e32696afe9991a7a4cff3196935ffc3692c301297b3d3b70fe"} Jul 26 17:57:05.271746 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:05.271424 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-t2h8t" Jul 26 17:57:05.303164 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:05.303111 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-t2h8t" podStartSLOduration=2.07663326 podStartE2EDuration="4.303095881s" totalImagesPullingTime="2.226462621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:57:05.301418071 +0000 UTC m=+479.254073772" watchObservedRunningTime="2026-07-26 17:57:05.303095881 +0000 UTC m=+479.255751569" Jul 26 17:57:06.387812 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:06.387780 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:06.624195 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:06.624162 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:09.029135 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:09.029090 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:09.183537 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:09.183502 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:12.262031 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:12.261997 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:13.297789 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:13.297749 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj"] Jul 26 17:57:13.305119 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:13.305097 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj" Jul 26 17:57:13.307464 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:13.307443 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rqdcd\"" Jul 26 17:57:13.320899 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:13.320874 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj"] Jul 26 17:57:13.409144 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:13.409107 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fd98c94-251f-4118-8167-9de1cbd399ed-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2cvcj\" (UID: \"5fd98c94-251f-4118-8167-9de1cbd399ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj" Jul 26 17:57:13.409335 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:13.409150 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzh4n\" (UniqueName: \"kubernetes.io/projected/5fd98c94-251f-4118-8167-9de1cbd399ed-kube-api-access-vzh4n\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2cvcj\" (UID: \"5fd98c94-251f-4118-8167-9de1cbd399ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj" Jul 26 17:57:13.509994 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:13.509940 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fd98c94-251f-4118-8167-9de1cbd399ed-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2cvcj\" (UID: \"5fd98c94-251f-4118-8167-9de1cbd399ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj" Jul 26 17:57:13.509994 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:13.509998 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzh4n\" (UniqueName: \"kubernetes.io/projected/5fd98c94-251f-4118-8167-9de1cbd399ed-kube-api-access-vzh4n\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2cvcj\" (UID: \"5fd98c94-251f-4118-8167-9de1cbd399ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj" Jul 26 17:57:13.510406 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:13.510388 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fd98c94-251f-4118-8167-9de1cbd399ed-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2cvcj\" (UID: \"5fd98c94-251f-4118-8167-9de1cbd399ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj" Jul 26 17:57:13.519382 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:13.519359 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzh4n\" (UniqueName: \"kubernetes.io/projected/5fd98c94-251f-4118-8167-9de1cbd399ed-kube-api-access-vzh4n\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2cvcj\" (UID: \"5fd98c94-251f-4118-8167-9de1cbd399ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj" Jul 26 17:57:13.615372 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:13.615287 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj" Jul 26 17:57:13.751313 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:13.751247 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj"] Jul 26 17:57:13.753643 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:57:13.753617 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fd98c94_251f_4118_8167_9de1cbd399ed.slice/crio-2648898dbd24b959e86222c46f57ad5b59825315a0304e4d8bb6779a0a3b8025 WatchSource:0}: Error finding container 2648898dbd24b959e86222c46f57ad5b59825315a0304e4d8bb6779a0a3b8025: Status 404 returned error can't find the container with id 2648898dbd24b959e86222c46f57ad5b59825315a0304e4d8bb6779a0a3b8025 Jul 26 17:57:14.307426 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:14.307390 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj" event={"ID":"5fd98c94-251f-4118-8167-9de1cbd399ed","Type":"ContainerStarted","Data":"2648898dbd24b959e86222c46f57ad5b59825315a0304e4d8bb6779a0a3b8025"} Jul 26 17:57:14.389367 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:14.389335 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:14.489444 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:14.489397 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:14.601952 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:14.601863 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:14.702533 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:14.702494 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:14.995195 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:14.995161 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:15.097580 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:15.097514 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:16.278707 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:16.278652 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-t2h8t" Jul 26 17:57:18.590307 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:18.590281 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:18.689477 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:18.689439 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:19.329995 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:19.329962 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj" event={"ID":"5fd98c94-251f-4118-8167-9de1cbd399ed","Type":"ContainerStarted","Data":"258da047c06d91eaf8143e83f01d1a1c8bee55677040b01fe360d0f12c24f97e"} Jul 26 17:57:19.330206 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:19.330049 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj" Jul 26 17:57:19.349923 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:19.349653 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj" podStartSLOduration=1.5787858240000001 podStartE2EDuration="6.34963392s" totalImagesPullingTime="4.770848096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:57:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:57:19.348861203 +0000 UTC m=+493.301516901" watchObservedRunningTime="2026-07-26 17:57:19.34963392 +0000 UTC m=+493.302289609" Jul 26 17:57:21.219338 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:21.219300 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:21.442597 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:21.442563 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:22.833232 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:22.833191 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:22.955117 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:22.955077 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:26.182999 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:26.182956 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:26.282538 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:26.282502 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:30.336963 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:30.336931 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj" Jul 26 17:57:32.037629 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.037593 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj"] Jul 26 17:57:32.038037 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.037803 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj" podUID="5fd98c94-251f-4118-8167-9de1cbd399ed" containerName="manager" containerID="cri-o://258da047c06d91eaf8143e83f01d1a1c8bee55677040b01fe360d0f12c24f97e" gracePeriod=2 Jul 26 17:57:32.043807 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.043777 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj"] Jul 26 17:57:32.054289 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.054259 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw"] Jul 26 17:57:32.054710 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.054694 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fd98c94-251f-4118-8167-9de1cbd399ed" containerName="manager" Jul 26 17:57:32.054785 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.054713 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fd98c94-251f-4118-8167-9de1cbd399ed" containerName="manager" Jul 26 17:57:32.054835 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.054794 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fd98c94-251f-4118-8167-9de1cbd399ed" containerName="manager" Jul 26 17:57:32.058002 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.057979 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw" Jul 26 17:57:32.060520 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:57:32.060475 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2cvcj\" is forbidden: User \"system:node:ip-10-0-128-209.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-209.ec2.internal' and this object" podUID="5fd98c94-251f-4118-8167-9de1cbd399ed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj" Jul 26 17:57:32.072624 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.072593 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw"] Jul 26 17:57:32.168028 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.167995 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qjqc\" (UniqueName: \"kubernetes.io/projected/acb04bb5-e292-41bc-8aa5-08eca27cf5ef-kube-api-access-7qjqc\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-j8rbw\" (UID: \"acb04bb5-e292-41bc-8aa5-08eca27cf5ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw" Jul 26 17:57:32.168174 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.168039 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/acb04bb5-e292-41bc-8aa5-08eca27cf5ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-j8rbw\" (UID: \"acb04bb5-e292-41bc-8aa5-08eca27cf5ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw" Jul 26 17:57:32.269106 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.269081 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qjqc\" (UniqueName: \"kubernetes.io/projected/acb04bb5-e292-41bc-8aa5-08eca27cf5ef-kube-api-access-7qjqc\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-j8rbw\" (UID: \"acb04bb5-e292-41bc-8aa5-08eca27cf5ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw" Jul 26 17:57:32.269239 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.269119 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/acb04bb5-e292-41bc-8aa5-08eca27cf5ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-j8rbw\" (UID: \"acb04bb5-e292-41bc-8aa5-08eca27cf5ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw" Jul 26 17:57:32.269450 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.269429 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/acb04bb5-e292-41bc-8aa5-08eca27cf5ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-j8rbw\" (UID: \"acb04bb5-e292-41bc-8aa5-08eca27cf5ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw" Jul 26 17:57:32.270113 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.270098 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj" Jul 26 17:57:32.271917 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:57:32.271895 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2cvcj\" is forbidden: User \"system:node:ip-10-0-128-209.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-209.ec2.internal' and this object" podUID="5fd98c94-251f-4118-8167-9de1cbd399ed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj" Jul 26 17:57:32.282690 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.282653 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qjqc\" (UniqueName: \"kubernetes.io/projected/acb04bb5-e292-41bc-8aa5-08eca27cf5ef-kube-api-access-7qjqc\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-j8rbw\" (UID: \"acb04bb5-e292-41bc-8aa5-08eca27cf5ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw" Jul 26 17:57:32.369722 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.369606 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fd98c94-251f-4118-8167-9de1cbd399ed-extensions-socket-volume\") pod \"5fd98c94-251f-4118-8167-9de1cbd399ed\" (UID: \"5fd98c94-251f-4118-8167-9de1cbd399ed\") " Jul 26 17:57:32.369722 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.369652 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzh4n\" (UniqueName: \"kubernetes.io/projected/5fd98c94-251f-4118-8167-9de1cbd399ed-kube-api-access-vzh4n\") pod \"5fd98c94-251f-4118-8167-9de1cbd399ed\" (UID: \"5fd98c94-251f-4118-8167-9de1cbd399ed\") " Jul 26 17:57:32.370216 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.370187 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fd98c94-251f-4118-8167-9de1cbd399ed-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5fd98c94-251f-4118-8167-9de1cbd399ed" (UID: "5fd98c94-251f-4118-8167-9de1cbd399ed"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 17:57:32.371726 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.371700 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fd98c94-251f-4118-8167-9de1cbd399ed-kube-api-access-vzh4n" (OuterVolumeSpecName: "kube-api-access-vzh4n") pod "5fd98c94-251f-4118-8167-9de1cbd399ed" (UID: "5fd98c94-251f-4118-8167-9de1cbd399ed"). InnerVolumeSpecName "kube-api-access-vzh4n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 17:57:32.387320 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.387290 2559 generic.go:356] "Generic (PLEG): container finished" podID="5fd98c94-251f-4118-8167-9de1cbd399ed" containerID="258da047c06d91eaf8143e83f01d1a1c8bee55677040b01fe360d0f12c24f97e" exitCode=0 Jul 26 17:57:32.387468 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.387341 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj" Jul 26 17:57:32.387468 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.387381 2559 scope.go:117] "RemoveContainer" containerID="258da047c06d91eaf8143e83f01d1a1c8bee55677040b01fe360d0f12c24f97e" Jul 26 17:57:32.389686 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:57:32.389651 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2cvcj\" is forbidden: User \"system:node:ip-10-0-128-209.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-209.ec2.internal' and this object" podUID="5fd98c94-251f-4118-8167-9de1cbd399ed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj" Jul 26 17:57:32.395607 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.395593 2559 scope.go:117] "RemoveContainer" containerID="258da047c06d91eaf8143e83f01d1a1c8bee55677040b01fe360d0f12c24f97e" Jul 26 17:57:32.395868 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:57:32.395846 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"258da047c06d91eaf8143e83f01d1a1c8bee55677040b01fe360d0f12c24f97e\": container with ID starting with 258da047c06d91eaf8143e83f01d1a1c8bee55677040b01fe360d0f12c24f97e not found: ID does not exist" containerID="258da047c06d91eaf8143e83f01d1a1c8bee55677040b01fe360d0f12c24f97e" Jul 26 17:57:32.395965 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.395876 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"258da047c06d91eaf8143e83f01d1a1c8bee55677040b01fe360d0f12c24f97e"} err="failed to get container status \"258da047c06d91eaf8143e83f01d1a1c8bee55677040b01fe360d0f12c24f97e\": rpc error: code = NotFound desc = could not find container \"258da047c06d91eaf8143e83f01d1a1c8bee55677040b01fe360d0f12c24f97e\": container with ID starting with 258da047c06d91eaf8143e83f01d1a1c8bee55677040b01fe360d0f12c24f97e not found: ID does not exist" Jul 26 17:57:32.396789 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:57:32.396770 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2cvcj\" is forbidden: User \"system:node:ip-10-0-128-209.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-209.ec2.internal' and this object" podUID="5fd98c94-251f-4118-8167-9de1cbd399ed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2cvcj" Jul 26 17:57:32.419951 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.419930 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw" Jul 26 17:57:32.443589 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.443560 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fd98c94-251f-4118-8167-9de1cbd399ed" path="/var/lib/kubelet/pods/5fd98c94-251f-4118-8167-9de1cbd399ed/volumes" Jul 26 17:57:32.470779 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.470733 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fd98c94-251f-4118-8167-9de1cbd399ed-extensions-socket-volume\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 17:57:32.470936 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.470838 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzh4n\" (UniqueName: \"kubernetes.io/projected/5fd98c94-251f-4118-8167-9de1cbd399ed-kube-api-access-vzh4n\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 17:57:32.557403 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:32.557369 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw"] Jul 26 17:57:32.559742 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:57:32.559716 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacb04bb5_e292_41bc_8aa5_08eca27cf5ef.slice/crio-7370bc12595a2b673dfdd7723506364f86a58d1f8045575faa9995685244703b WatchSource:0}: Error finding container 7370bc12595a2b673dfdd7723506364f86a58d1f8045575faa9995685244703b: Status 404 returned error can't find the container with id 7370bc12595a2b673dfdd7723506364f86a58d1f8045575faa9995685244703b Jul 26 17:57:33.393393 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:33.393360 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw" event={"ID":"acb04bb5-e292-41bc-8aa5-08eca27cf5ef","Type":"ContainerStarted","Data":"80ba04a7b95ba1d71e8a0b3de1adc99e11abc848794b6681c2da385f48d3aee6"} Jul 26 17:57:33.393814 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:33.393411 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw" event={"ID":"acb04bb5-e292-41bc-8aa5-08eca27cf5ef","Type":"ContainerStarted","Data":"7370bc12595a2b673dfdd7723506364f86a58d1f8045575faa9995685244703b"} Jul 26 17:57:33.393814 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:33.393486 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw" Jul 26 17:57:33.415144 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:33.415091 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw" podStartSLOduration=1.415076121 podStartE2EDuration="1.415076121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:57:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:57:33.412920973 +0000 UTC m=+507.365576680" watchObservedRunningTime="2026-07-26 17:57:33.415076121 +0000 UTC m=+507.367731809" Jul 26 17:57:34.244545 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:34.244511 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:34.351791 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:34.351749 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:34.829353 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:34.829310 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:34.929494 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:34.929444 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:37.190866 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:37.190832 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:37.291351 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:37.291316 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:37.825136 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:37.825096 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:37.925875 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:37.925842 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:42.434294 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:42.434254 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:42.533136 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:42.533104 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:44.399026 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:44.398996 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw" Jul 26 17:57:44.661224 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:44.661142 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76"] Jul 26 17:57:44.669439 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:44.669411 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76" Jul 26 17:57:44.680248 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:44.680152 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76"] Jul 26 17:57:44.780570 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:44.780540 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d262968-3507-422b-977f-ccf2d69fedfd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tnf76\" (UID: \"0d262968-3507-422b-977f-ccf2d69fedfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76" Jul 26 17:57:44.780570 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:44.780579 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmnjt\" (UniqueName: \"kubernetes.io/projected/0d262968-3507-422b-977f-ccf2d69fedfd-kube-api-access-rmnjt\") pod \"kuadrant-operator-controller-manager-86878c769f-tnf76\" (UID: \"0d262968-3507-422b-977f-ccf2d69fedfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76" Jul 26 17:57:44.881180 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:44.881140 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d262968-3507-422b-977f-ccf2d69fedfd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tnf76\" (UID: \"0d262968-3507-422b-977f-ccf2d69fedfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76" Jul 26 17:57:44.881180 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:44.881186 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmnjt\" (UniqueName: \"kubernetes.io/projected/0d262968-3507-422b-977f-ccf2d69fedfd-kube-api-access-rmnjt\") pod \"kuadrant-operator-controller-manager-86878c769f-tnf76\" (UID: \"0d262968-3507-422b-977f-ccf2d69fedfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76" Jul 26 17:57:44.881527 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:44.881507 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d262968-3507-422b-977f-ccf2d69fedfd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tnf76\" (UID: \"0d262968-3507-422b-977f-ccf2d69fedfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76" Jul 26 17:57:44.892556 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:44.892527 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmnjt\" (UniqueName: \"kubernetes.io/projected/0d262968-3507-422b-977f-ccf2d69fedfd-kube-api-access-rmnjt\") pod \"kuadrant-operator-controller-manager-86878c769f-tnf76\" (UID: \"0d262968-3507-422b-977f-ccf2d69fedfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76" Jul 26 17:57:44.980456 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:44.980429 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76" Jul 26 17:57:45.045151 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:45.045112 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:45.116567 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:45.116543 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76"] Jul 26 17:57:45.118292 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:57:45.118263 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d262968_3507_422b_977f_ccf2d69fedfd.slice/crio-a325788ce5df17292a5129717fb0b4adfee20ed5268ed26cac4fa73a8ba24df7 WatchSource:0}: Error finding container a325788ce5df17292a5129717fb0b4adfee20ed5268ed26cac4fa73a8ba24df7: Status 404 returned error can't find the container with id a325788ce5df17292a5129717fb0b4adfee20ed5268ed26cac4fa73a8ba24df7 Jul 26 17:57:45.119441 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:45.119413 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:57:45.443286 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:45.443192 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76" event={"ID":"0d262968-3507-422b-977f-ccf2d69fedfd","Type":"ContainerStarted","Data":"758a41aee07fbe02bcd79a7146a9f7e448f1114b62d2442c4c867e5817db5618"} Jul 26 17:57:45.443286 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:45.443239 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76" event={"ID":"0d262968-3507-422b-977f-ccf2d69fedfd","Type":"ContainerStarted","Data":"a325788ce5df17292a5129717fb0b4adfee20ed5268ed26cac4fa73a8ba24df7"} Jul 26 17:57:45.443774 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:45.443288 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76" Jul 26 17:57:45.468292 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:45.468237 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76" podStartSLOduration=1.468219497 podStartE2EDuration="1.468219497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:57:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:57:45.467511939 +0000 UTC m=+519.420167671" watchObservedRunningTime="2026-07-26 17:57:45.468219497 +0000 UTC m=+519.420875184" Jul 26 17:57:56.448857 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:56.448824 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76" Jul 26 17:57:56.498216 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:56.498182 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw"] Jul 26 17:57:56.498419 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:56.498395 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw" podUID="acb04bb5-e292-41bc-8aa5-08eca27cf5ef" containerName="manager" containerID="cri-o://80ba04a7b95ba1d71e8a0b3de1adc99e11abc848794b6681c2da385f48d3aee6" gracePeriod=10 Jul 26 17:57:56.739841 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:56.739817 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw" Jul 26 17:57:56.884155 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:56.884111 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qjqc\" (UniqueName: \"kubernetes.io/projected/acb04bb5-e292-41bc-8aa5-08eca27cf5ef-kube-api-access-7qjqc\") pod \"acb04bb5-e292-41bc-8aa5-08eca27cf5ef\" (UID: \"acb04bb5-e292-41bc-8aa5-08eca27cf5ef\") " Jul 26 17:57:56.884345 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:56.884173 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/acb04bb5-e292-41bc-8aa5-08eca27cf5ef-extensions-socket-volume\") pod \"acb04bb5-e292-41bc-8aa5-08eca27cf5ef\" (UID: \"acb04bb5-e292-41bc-8aa5-08eca27cf5ef\") " Jul 26 17:57:56.884559 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:56.884534 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acb04bb5-e292-41bc-8aa5-08eca27cf5ef-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "acb04bb5-e292-41bc-8aa5-08eca27cf5ef" (UID: "acb04bb5-e292-41bc-8aa5-08eca27cf5ef"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 17:57:56.886219 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:56.886194 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acb04bb5-e292-41bc-8aa5-08eca27cf5ef-kube-api-access-7qjqc" (OuterVolumeSpecName: "kube-api-access-7qjqc") pod "acb04bb5-e292-41bc-8aa5-08eca27cf5ef" (UID: "acb04bb5-e292-41bc-8aa5-08eca27cf5ef"). InnerVolumeSpecName "kube-api-access-7qjqc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 17:57:56.985279 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:56.985179 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qjqc\" (UniqueName: \"kubernetes.io/projected/acb04bb5-e292-41bc-8aa5-08eca27cf5ef-kube-api-access-7qjqc\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 17:57:56.985279 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:56.985223 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/acb04bb5-e292-41bc-8aa5-08eca27cf5ef-extensions-socket-volume\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 17:57:57.499598 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:57.499564 2559 generic.go:356] "Generic (PLEG): container finished" podID="acb04bb5-e292-41bc-8aa5-08eca27cf5ef" containerID="80ba04a7b95ba1d71e8a0b3de1adc99e11abc848794b6681c2da385f48d3aee6" exitCode=0 Jul 26 17:57:57.500044 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:57.499623 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw" Jul 26 17:57:57.500044 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:57.499662 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw" event={"ID":"acb04bb5-e292-41bc-8aa5-08eca27cf5ef","Type":"ContainerDied","Data":"80ba04a7b95ba1d71e8a0b3de1adc99e11abc848794b6681c2da385f48d3aee6"} Jul 26 17:57:57.500044 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:57.499744 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw" event={"ID":"acb04bb5-e292-41bc-8aa5-08eca27cf5ef","Type":"ContainerDied","Data":"7370bc12595a2b673dfdd7723506364f86a58d1f8045575faa9995685244703b"} Jul 26 17:57:57.500044 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:57.499773 2559 scope.go:117] "RemoveContainer" containerID="80ba04a7b95ba1d71e8a0b3de1adc99e11abc848794b6681c2da385f48d3aee6" Jul 26 17:57:57.508466 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:57.508445 2559 scope.go:117] "RemoveContainer" containerID="80ba04a7b95ba1d71e8a0b3de1adc99e11abc848794b6681c2da385f48d3aee6" Jul 26 17:57:57.508736 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:57:57.508717 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80ba04a7b95ba1d71e8a0b3de1adc99e11abc848794b6681c2da385f48d3aee6\": container with ID starting with 80ba04a7b95ba1d71e8a0b3de1adc99e11abc848794b6681c2da385f48d3aee6 not found: ID does not exist" containerID="80ba04a7b95ba1d71e8a0b3de1adc99e11abc848794b6681c2da385f48d3aee6" Jul 26 17:57:57.508783 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:57.508746 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80ba04a7b95ba1d71e8a0b3de1adc99e11abc848794b6681c2da385f48d3aee6"} err="failed to get container status \"80ba04a7b95ba1d71e8a0b3de1adc99e11abc848794b6681c2da385f48d3aee6\": rpc error: code = NotFound desc = could not find container \"80ba04a7b95ba1d71e8a0b3de1adc99e11abc848794b6681c2da385f48d3aee6\": container with ID starting with 80ba04a7b95ba1d71e8a0b3de1adc99e11abc848794b6681c2da385f48d3aee6 not found: ID does not exist" Jul 26 17:57:57.528693 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:57.528644 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw"] Jul 26 17:57:57.532995 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:57.532965 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-j8rbw"] Jul 26 17:57:58.438631 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:57:58.438557 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acb04bb5-e292-41bc-8aa5-08eca27cf5ef" path="/var/lib/kubelet/pods/acb04bb5-e292-41bc-8aa5-08eca27cf5ef/volumes" Jul 26 17:58:00.821145 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:00.821109 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:01.201290 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:01.201205 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:01.978293 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:01.978257 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:06.425976 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:06.425935 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:06.481017 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:06.480990 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:07.461813 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:07.461776 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:07.516070 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:07.516028 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:15.149609 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:15.149573 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:15.259215 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:15.259179 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:15.327479 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:15.327444 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:15.990817 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:15.990782 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:16.090149 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:16.090114 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:34.218898 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:34.218857 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:34.316112 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:34.316075 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:34.673633 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:34.673555 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-qtk7x"] Jul 26 17:58:34.673878 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:34.673864 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acb04bb5-e292-41bc-8aa5-08eca27cf5ef" containerName="manager" Jul 26 17:58:34.673943 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:34.673879 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="acb04bb5-e292-41bc-8aa5-08eca27cf5ef" containerName="manager" Jul 26 17:58:34.673998 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:34.673948 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="acb04bb5-e292-41bc-8aa5-08eca27cf5ef" containerName="manager" Jul 26 17:58:34.676998 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:34.676977 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qtk7x" Jul 26 17:58:34.679628 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:34.679604 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 26 17:58:34.679759 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:34.679603 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 26 17:58:34.680556 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:34.680534 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-lnrfs\"" Jul 26 17:58:34.680556 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:34.680552 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 26 17:58:34.686425 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:34.686402 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-qtk7x"] Jul 26 17:58:34.713448 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:34.713414 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:34.774505 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:34.774472 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/07edbc3f-69bc-4c38-ad7d-38b81abac5c9-data\") pod \"postgres-69fb4594cf-qtk7x\" (UID: \"07edbc3f-69bc-4c38-ad7d-38b81abac5c9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qtk7x" Jul 26 17:58:34.774718 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:34.774526 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttqbj\" (UniqueName: \"kubernetes.io/projected/07edbc3f-69bc-4c38-ad7d-38b81abac5c9-kube-api-access-ttqbj\") pod \"postgres-69fb4594cf-qtk7x\" (UID: \"07edbc3f-69bc-4c38-ad7d-38b81abac5c9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qtk7x" Jul 26 17:58:34.875612 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:34.875576 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/07edbc3f-69bc-4c38-ad7d-38b81abac5c9-data\") pod \"postgres-69fb4594cf-qtk7x\" (UID: \"07edbc3f-69bc-4c38-ad7d-38b81abac5c9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qtk7x" Jul 26 17:58:34.875832 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:34.875645 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttqbj\" (UniqueName: \"kubernetes.io/projected/07edbc3f-69bc-4c38-ad7d-38b81abac5c9-kube-api-access-ttqbj\") pod \"postgres-69fb4594cf-qtk7x\" (UID: \"07edbc3f-69bc-4c38-ad7d-38b81abac5c9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qtk7x" Jul 26 17:58:34.876001 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:34.875978 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/07edbc3f-69bc-4c38-ad7d-38b81abac5c9-data\") pod \"postgres-69fb4594cf-qtk7x\" (UID: \"07edbc3f-69bc-4c38-ad7d-38b81abac5c9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qtk7x" Jul 26 17:58:34.891574 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:34.891553 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttqbj\" (UniqueName: \"kubernetes.io/projected/07edbc3f-69bc-4c38-ad7d-38b81abac5c9-kube-api-access-ttqbj\") pod \"postgres-69fb4594cf-qtk7x\" (UID: \"07edbc3f-69bc-4c38-ad7d-38b81abac5c9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qtk7x" Jul 26 17:58:34.988470 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:34.988438 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qtk7x" Jul 26 17:58:35.118749 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:35.118714 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-qtk7x"] Jul 26 17:58:35.122263 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:58:35.122238 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07edbc3f_69bc_4c38_ad7d_38b81abac5c9.slice/crio-c76436220b22acc439640704f8e6569e9c2974740b5d11526963b2d972f30ecc WatchSource:0}: Error finding container c76436220b22acc439640704f8e6569e9c2974740b5d11526963b2d972f30ecc: Status 404 returned error can't find the container with id c76436220b22acc439640704f8e6569e9c2974740b5d11526963b2d972f30ecc Jul 26 17:58:35.662225 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:35.662189 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qtk7x" event={"ID":"07edbc3f-69bc-4c38-ad7d-38b81abac5c9","Type":"ContainerStarted","Data":"c76436220b22acc439640704f8e6569e9c2974740b5d11526963b2d972f30ecc"} Jul 26 17:58:40.684580 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:40.684540 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qtk7x" event={"ID":"07edbc3f-69bc-4c38-ad7d-38b81abac5c9","Type":"ContainerStarted","Data":"5a05b3a08c440497f9927a86c2a6163e6d27ae4070afcd87ed362fe9a17d0a8f"} Jul 26 17:58:40.685025 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:40.684681 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qtk7x" Jul 26 17:58:40.703499 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:40.703454 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qtk7x" podStartSLOduration=1.9974189629999999 podStartE2EDuration="6.703439551s" totalImagesPullingTime="4.706020588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:58:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:58:40.702758179 +0000 UTC m=+574.655413878" watchObservedRunningTime="2026-07-26 17:58:40.703439551 +0000 UTC m=+574.656095239" Jul 26 17:58:44.086189 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:44.086129 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:44.144192 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:44.144157 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:46.716650 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:46.716621 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qtk7x" Jul 26 17:58:47.011576 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:47.011474 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:47.201606 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:47.201573 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:53.256895 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.256852 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:53.260972 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.260947 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-584bfb64d4-llxdq"] Jul 26 17:58:53.286311 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.286276 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-584bfb64d4-llxdq"] Jul 26 17:58:53.286484 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.286317 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-584bfb64d4-llxdq" Jul 26 17:58:53.289640 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.289616 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 26 17:58:53.289985 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.289970 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 26 17:58:53.290248 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.290234 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ccp5l\"" Jul 26 17:58:53.328176 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.328138 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:53.379999 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.379959 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:53.433605 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.433573 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67b95f7b6f-c8mtn"] Jul 26 17:58:53.437274 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.437252 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b95f7b6f-c8mtn" Jul 26 17:58:53.445995 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.445961 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b95f7b6f-c8mtn"] Jul 26 17:58:53.447081 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.447059 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b44305c4-192e-4440-88c2-a129a025b1da-webhook-cert\") pod \"maas-controller-584bfb64d4-llxdq\" (UID: \"b44305c4-192e-4440-88c2-a129a025b1da\") " pod="opendatahub/maas-controller-584bfb64d4-llxdq" Jul 26 17:58:53.447269 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.447116 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nmv4\" (UniqueName: \"kubernetes.io/projected/b44305c4-192e-4440-88c2-a129a025b1da-kube-api-access-4nmv4\") pod \"maas-controller-584bfb64d4-llxdq\" (UID: \"b44305c4-192e-4440-88c2-a129a025b1da\") " pod="opendatahub/maas-controller-584bfb64d4-llxdq" Jul 26 17:58:53.524892 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.524784 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:53.548293 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.548257 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b44305c4-192e-4440-88c2-a129a025b1da-webhook-cert\") pod \"maas-controller-584bfb64d4-llxdq\" (UID: \"b44305c4-192e-4440-88c2-a129a025b1da\") " pod="opendatahub/maas-controller-584bfb64d4-llxdq" Jul 26 17:58:53.548469 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.548327 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nmv4\" (UniqueName: \"kubernetes.io/projected/b44305c4-192e-4440-88c2-a129a025b1da-kube-api-access-4nmv4\") pod \"maas-controller-584bfb64d4-llxdq\" (UID: \"b44305c4-192e-4440-88c2-a129a025b1da\") " pod="opendatahub/maas-controller-584bfb64d4-llxdq" Jul 26 17:58:53.548469 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.548364 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f048d6a1-bf88-454f-873e-6f0c97259f06-webhook-cert\") pod \"maas-controller-67b95f7b6f-c8mtn\" (UID: \"f048d6a1-bf88-454f-873e-6f0c97259f06\") " pod="opendatahub/maas-controller-67b95f7b6f-c8mtn" Jul 26 17:58:53.548593 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.548459 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfrhn\" (UniqueName: \"kubernetes.io/projected/f048d6a1-bf88-454f-873e-6f0c97259f06-kube-api-access-mfrhn\") pod \"maas-controller-67b95f7b6f-c8mtn\" (UID: \"f048d6a1-bf88-454f-873e-6f0c97259f06\") " pod="opendatahub/maas-controller-67b95f7b6f-c8mtn" Jul 26 17:58:53.550837 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.550813 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b44305c4-192e-4440-88c2-a129a025b1da-webhook-cert\") pod \"maas-controller-584bfb64d4-llxdq\" (UID: \"b44305c4-192e-4440-88c2-a129a025b1da\") " pod="opendatahub/maas-controller-584bfb64d4-llxdq" Jul 26 17:58:53.557043 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.557019 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nmv4\" (UniqueName: \"kubernetes.io/projected/b44305c4-192e-4440-88c2-a129a025b1da-kube-api-access-4nmv4\") pod \"maas-controller-584bfb64d4-llxdq\" (UID: \"b44305c4-192e-4440-88c2-a129a025b1da\") " pod="opendatahub/maas-controller-584bfb64d4-llxdq" Jul 26 17:58:53.599559 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.599524 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-584bfb64d4-llxdq" Jul 26 17:58:53.649975 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.649937 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f048d6a1-bf88-454f-873e-6f0c97259f06-webhook-cert\") pod \"maas-controller-67b95f7b6f-c8mtn\" (UID: \"f048d6a1-bf88-454f-873e-6f0c97259f06\") " pod="opendatahub/maas-controller-67b95f7b6f-c8mtn" Jul 26 17:58:53.650127 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.650009 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfrhn\" (UniqueName: \"kubernetes.io/projected/f048d6a1-bf88-454f-873e-6f0c97259f06-kube-api-access-mfrhn\") pod \"maas-controller-67b95f7b6f-c8mtn\" (UID: \"f048d6a1-bf88-454f-873e-6f0c97259f06\") " pod="opendatahub/maas-controller-67b95f7b6f-c8mtn" Jul 26 17:58:53.653109 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.653068 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f048d6a1-bf88-454f-873e-6f0c97259f06-webhook-cert\") pod \"maas-controller-67b95f7b6f-c8mtn\" (UID: \"f048d6a1-bf88-454f-873e-6f0c97259f06\") " pod="opendatahub/maas-controller-67b95f7b6f-c8mtn" Jul 26 17:58:53.660032 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.659528 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfrhn\" (UniqueName: \"kubernetes.io/projected/f048d6a1-bf88-454f-873e-6f0c97259f06-kube-api-access-mfrhn\") pod \"maas-controller-67b95f7b6f-c8mtn\" (UID: \"f048d6a1-bf88-454f-873e-6f0c97259f06\") " pod="opendatahub/maas-controller-67b95f7b6f-c8mtn" Jul 26 17:58:53.728690 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.728648 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-584bfb64d4-llxdq"] Jul 26 17:58:53.731120 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:58:53.731097 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb44305c4_192e_4440_88c2_a129a025b1da.slice/crio-26b43a9929726c9abbd272641ac9b16c68e2527502735c6069bb95406527dd33 WatchSource:0}: Error finding container 26b43a9929726c9abbd272641ac9b16c68e2527502735c6069bb95406527dd33: Status 404 returned error can't find the container with id 26b43a9929726c9abbd272641ac9b16c68e2527502735c6069bb95406527dd33 Jul 26 17:58:53.743070 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.743047 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-584bfb64d4-llxdq" event={"ID":"b44305c4-192e-4440-88c2-a129a025b1da","Type":"ContainerStarted","Data":"26b43a9929726c9abbd272641ac9b16c68e2527502735c6069bb95406527dd33"} Jul 26 17:58:53.748206 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.748189 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b95f7b6f-c8mtn" Jul 26 17:58:53.876886 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:53.876859 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b95f7b6f-c8mtn"] Jul 26 17:58:53.879175 ip-10-0-128-209 kubenswrapper[2559]: W0726 17:58:53.879134 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf048d6a1_bf88_454f_873e_6f0c97259f06.slice/crio-0c352f9d096d7f33845978935e47f652ca503d9321f204bf15d38d67771752b0 WatchSource:0}: Error finding container 0c352f9d096d7f33845978935e47f652ca503d9321f204bf15d38d67771752b0: Status 404 returned error can't find the container with id 0c352f9d096d7f33845978935e47f652ca503d9321f204bf15d38d67771752b0 Jul 26 17:58:54.752237 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:54.752189 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b95f7b6f-c8mtn" event={"ID":"f048d6a1-bf88-454f-873e-6f0c97259f06","Type":"ContainerStarted","Data":"0c352f9d096d7f33845978935e47f652ca503d9321f204bf15d38d67771752b0"} Jul 26 17:58:56.330071 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:56.330019 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:56.429346 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:56.429261 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:58:56.762365 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:56.762329 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b95f7b6f-c8mtn" event={"ID":"f048d6a1-bf88-454f-873e-6f0c97259f06","Type":"ContainerStarted","Data":"b32293cbad2cf82e9ac753df3eae42371588d15d7e1229ad657570cfe9df4bb8"} Jul 26 17:58:56.762585 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:56.762439 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67b95f7b6f-c8mtn" Jul 26 17:58:56.763752 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:56.763726 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-584bfb64d4-llxdq" event={"ID":"b44305c4-192e-4440-88c2-a129a025b1da","Type":"ContainerStarted","Data":"fd8f3c579a779ab8ca3b54f7092fba23aa628ce859d0ef79bd02f90e838be402"} Jul 26 17:58:56.763880 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:56.763864 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-584bfb64d4-llxdq" Jul 26 17:58:56.781522 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:56.781473 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67b95f7b6f-c8mtn" podStartSLOduration=1.499281275 podStartE2EDuration="3.781458765s" totalImagesPullingTime="2.28217749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:58:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:58:56.78059396 +0000 UTC m=+590.733249649" watchObservedRunningTime="2026-07-26 17:58:56.781458765 +0000 UTC m=+590.734114452" Jul 26 17:58:56.801547 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:58:56.801489 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-584bfb64d4-llxdq" podStartSLOduration=1.377270416 podStartE2EDuration="3.80147436s" totalImagesPullingTime="2.424203944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:58:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:58:56.79977976 +0000 UTC m=+590.752435471" watchObservedRunningTime="2026-07-26 17:58:56.80147436 +0000 UTC m=+590.754130047" Jul 26 17:59:01.575390 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:01.575361 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:59:01.640908 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:01.640875 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:59:07.770154 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:07.770123 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-584bfb64d4-llxdq" Jul 26 17:59:07.772447 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:07.770205 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67b95f7b6f-c8mtn" Jul 26 17:59:07.832434 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:07.832399 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-584bfb64d4-llxdq"] Jul 26 17:59:07.832616 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:07.832577 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-584bfb64d4-llxdq" podUID="b44305c4-192e-4440-88c2-a129a025b1da" containerName="manager" containerID="cri-o://fd8f3c579a779ab8ca3b54f7092fba23aa628ce859d0ef79bd02f90e838be402" gracePeriod=30 Jul 26 17:59:08.069082 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:08.069057 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-584bfb64d4-llxdq" Jul 26 17:59:08.172427 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:08.172392 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b44305c4-192e-4440-88c2-a129a025b1da-webhook-cert\") pod \"b44305c4-192e-4440-88c2-a129a025b1da\" (UID: \"b44305c4-192e-4440-88c2-a129a025b1da\") " Jul 26 17:59:08.172427 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:08.172432 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nmv4\" (UniqueName: \"kubernetes.io/projected/b44305c4-192e-4440-88c2-a129a025b1da-kube-api-access-4nmv4\") pod \"b44305c4-192e-4440-88c2-a129a025b1da\" (UID: \"b44305c4-192e-4440-88c2-a129a025b1da\") " Jul 26 17:59:08.174641 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:08.174612 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b44305c4-192e-4440-88c2-a129a025b1da-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b44305c4-192e-4440-88c2-a129a025b1da" (UID: "b44305c4-192e-4440-88c2-a129a025b1da"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 26 17:59:08.174759 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:08.174687 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b44305c4-192e-4440-88c2-a129a025b1da-kube-api-access-4nmv4" (OuterVolumeSpecName: "kube-api-access-4nmv4") pod "b44305c4-192e-4440-88c2-a129a025b1da" (UID: "b44305c4-192e-4440-88c2-a129a025b1da"). InnerVolumeSpecName "kube-api-access-4nmv4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 17:59:08.272983 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:08.272946 2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b44305c4-192e-4440-88c2-a129a025b1da-webhook-cert\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 17:59:08.272983 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:08.272974 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nmv4\" (UniqueName: \"kubernetes.io/projected/b44305c4-192e-4440-88c2-a129a025b1da-kube-api-access-4nmv4\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 17:59:08.814583 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:08.814543 2559 generic.go:356] "Generic (PLEG): container finished" podID="b44305c4-192e-4440-88c2-a129a025b1da" containerID="fd8f3c579a779ab8ca3b54f7092fba23aa628ce859d0ef79bd02f90e838be402" exitCode=0 Jul 26 17:59:08.814997 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:08.814591 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-584bfb64d4-llxdq" event={"ID":"b44305c4-192e-4440-88c2-a129a025b1da","Type":"ContainerDied","Data":"fd8f3c579a779ab8ca3b54f7092fba23aa628ce859d0ef79bd02f90e838be402"} Jul 26 17:59:08.814997 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:08.814634 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-584bfb64d4-llxdq" event={"ID":"b44305c4-192e-4440-88c2-a129a025b1da","Type":"ContainerDied","Data":"26b43a9929726c9abbd272641ac9b16c68e2527502735c6069bb95406527dd33"} Jul 26 17:59:08.814997 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:08.814659 2559 scope.go:117] "RemoveContainer" containerID="fd8f3c579a779ab8ca3b54f7092fba23aa628ce859d0ef79bd02f90e838be402" Jul 26 17:59:08.814997 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:08.814595 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-584bfb64d4-llxdq" Jul 26 17:59:08.822538 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:08.822519 2559 scope.go:117] "RemoveContainer" containerID="fd8f3c579a779ab8ca3b54f7092fba23aa628ce859d0ef79bd02f90e838be402" Jul 26 17:59:08.822814 ip-10-0-128-209 kubenswrapper[2559]: E0726 17:59:08.822796 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd8f3c579a779ab8ca3b54f7092fba23aa628ce859d0ef79bd02f90e838be402\": container with ID starting with fd8f3c579a779ab8ca3b54f7092fba23aa628ce859d0ef79bd02f90e838be402 not found: ID does not exist" containerID="fd8f3c579a779ab8ca3b54f7092fba23aa628ce859d0ef79bd02f90e838be402" Jul 26 17:59:08.822893 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:08.822822 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd8f3c579a779ab8ca3b54f7092fba23aa628ce859d0ef79bd02f90e838be402"} err="failed to get container status \"fd8f3c579a779ab8ca3b54f7092fba23aa628ce859d0ef79bd02f90e838be402\": rpc error: code = NotFound desc = could not find container \"fd8f3c579a779ab8ca3b54f7092fba23aa628ce859d0ef79bd02f90e838be402\": container with ID starting with fd8f3c579a779ab8ca3b54f7092fba23aa628ce859d0ef79bd02f90e838be402 not found: ID does not exist" Jul 26 17:59:08.831337 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:08.831311 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-584bfb64d4-llxdq"] Jul 26 17:59:08.832705 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:08.832687 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-584bfb64d4-llxdq"] Jul 26 17:59:10.437890 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:10.437861 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b44305c4-192e-4440-88c2-a129a025b1da" path="/var/lib/kubelet/pods/b44305c4-192e-4440-88c2-a129a025b1da/volumes" Jul 26 17:59:30.123321 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:30.123229 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:59:31.310126 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:31.310088 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:59:31.445005 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:31.444975 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:59:41.904990 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:41.904957 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:59:42.020696 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:42.020633 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 17:59:57.622555 ip-10-0-128-209 kubenswrapper[2559]: I0726 17:59:57.622516 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:00:09.407197 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.407149 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77"] Jul 26 18:00:09.407654 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.407613 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b44305c4-192e-4440-88c2-a129a025b1da" containerName="manager" Jul 26 18:00:09.407654 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.407628 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b44305c4-192e-4440-88c2-a129a025b1da" containerName="manager" Jul 26 18:00:09.407763 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.407745 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b44305c4-192e-4440-88c2-a129a025b1da" containerName="manager" Jul 26 18:00:09.412355 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.412327 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:09.414989 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.414970 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 26 18:00:09.415978 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.415958 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 26 18:00:09.416228 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.416213 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 26 18:00:09.416341 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.416321 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-8klrd\"" Jul 26 18:00:09.420563 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.420542 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77"] Jul 26 18:00:09.441495 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.441457 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:00:09.487357 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.487321 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhp6q\" (UniqueName: \"kubernetes.io/projected/d9120c21-cdb3-42ae-b0f5-58b546a63710-kube-api-access-jhp6q\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8x77\" (UID: \"d9120c21-cdb3-42ae-b0f5-58b546a63710\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:09.487529 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.487370 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9120c21-cdb3-42ae-b0f5-58b546a63710-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8x77\" (UID: \"d9120c21-cdb3-42ae-b0f5-58b546a63710\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:09.487529 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.487463 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9120c21-cdb3-42ae-b0f5-58b546a63710-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8x77\" (UID: \"d9120c21-cdb3-42ae-b0f5-58b546a63710\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:09.487529 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.487522 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9120c21-cdb3-42ae-b0f5-58b546a63710-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8x77\" (UID: \"d9120c21-cdb3-42ae-b0f5-58b546a63710\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:09.487688 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.487552 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9120c21-cdb3-42ae-b0f5-58b546a63710-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8x77\" (UID: \"d9120c21-cdb3-42ae-b0f5-58b546a63710\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:09.487688 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.487571 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9120c21-cdb3-42ae-b0f5-58b546a63710-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8x77\" (UID: \"d9120c21-cdb3-42ae-b0f5-58b546a63710\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:09.588484 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.588451 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9120c21-cdb3-42ae-b0f5-58b546a63710-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8x77\" (UID: \"d9120c21-cdb3-42ae-b0f5-58b546a63710\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:09.588484 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.588490 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9120c21-cdb3-42ae-b0f5-58b546a63710-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8x77\" (UID: \"d9120c21-cdb3-42ae-b0f5-58b546a63710\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:09.588734 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.588510 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9120c21-cdb3-42ae-b0f5-58b546a63710-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8x77\" (UID: \"d9120c21-cdb3-42ae-b0f5-58b546a63710\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:09.588734 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.588552 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhp6q\" (UniqueName: \"kubernetes.io/projected/d9120c21-cdb3-42ae-b0f5-58b546a63710-kube-api-access-jhp6q\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8x77\" (UID: \"d9120c21-cdb3-42ae-b0f5-58b546a63710\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:09.588734 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.588589 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9120c21-cdb3-42ae-b0f5-58b546a63710-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8x77\" (UID: \"d9120c21-cdb3-42ae-b0f5-58b546a63710\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:09.588734 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.588645 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9120c21-cdb3-42ae-b0f5-58b546a63710-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8x77\" (UID: \"d9120c21-cdb3-42ae-b0f5-58b546a63710\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:09.588994 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.588940 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9120c21-cdb3-42ae-b0f5-58b546a63710-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8x77\" (UID: \"d9120c21-cdb3-42ae-b0f5-58b546a63710\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:09.588994 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.588980 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9120c21-cdb3-42ae-b0f5-58b546a63710-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8x77\" (UID: \"d9120c21-cdb3-42ae-b0f5-58b546a63710\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:09.589175 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.589155 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9120c21-cdb3-42ae-b0f5-58b546a63710-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8x77\" (UID: \"d9120c21-cdb3-42ae-b0f5-58b546a63710\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:09.590978 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.590955 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9120c21-cdb3-42ae-b0f5-58b546a63710-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8x77\" (UID: \"d9120c21-cdb3-42ae-b0f5-58b546a63710\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:09.591178 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.591161 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9120c21-cdb3-42ae-b0f5-58b546a63710-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8x77\" (UID: \"d9120c21-cdb3-42ae-b0f5-58b546a63710\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:09.596416 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.596395 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhp6q\" (UniqueName: \"kubernetes.io/projected/d9120c21-cdb3-42ae-b0f5-58b546a63710-kube-api-access-jhp6q\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8x77\" (UID: \"d9120c21-cdb3-42ae-b0f5-58b546a63710\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:09.723760 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.723724 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:09.853972 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.853827 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77"] Jul 26 18:00:09.856133 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:00:09.856100 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9120c21_cdb3_42ae_b0f5_58b546a63710.slice/crio-b2225d288165442211fe1b2329ad87901d14bcccbea3c0c2bc0ba8ffaf933689 WatchSource:0}: Error finding container b2225d288165442211fe1b2329ad87901d14bcccbea3c0c2bc0ba8ffaf933689: Status 404 returned error can't find the container with id b2225d288165442211fe1b2329ad87901d14bcccbea3c0c2bc0ba8ffaf933689 Jul 26 18:00:09.857449 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:09.857428 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 26 18:00:10.075235 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:10.075153 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" event={"ID":"d9120c21-cdb3-42ae-b0f5-58b546a63710","Type":"ContainerStarted","Data":"b2225d288165442211fe1b2329ad87901d14bcccbea3c0c2bc0ba8ffaf933689"} Jul 26 18:00:16.103698 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:16.103654 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" event={"ID":"d9120c21-cdb3-42ae-b0f5-58b546a63710","Type":"ContainerStarted","Data":"528eaae2802a42487e80a7f2c5acb3feeabd8ab3f594c34847ff2e372e7506ac"} Jul 26 18:00:16.121302 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:16.121243 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" podStartSLOduration=1.295610503 podStartE2EDuration="7.121227116s" totalImagesPullingTime="5.825616613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:00:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:00:16.119921831 +0000 UTC m=+670.072577547" watchObservedRunningTime="2026-07-26 18:00:16.121227116 +0000 UTC m=+670.073882803" Jul 26 18:00:19.724625 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:19.724582 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:19.725024 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:19.724651 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:19.737383 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:19.737354 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:20.132445 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:20.132362 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8x77" Jul 26 18:00:28.687764 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:28.687689 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:00:34.386479 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.386448 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq"] Jul 26 18:00:34.390967 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.390945 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:34.393455 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.393432 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 26 18:00:34.400213 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.400187 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq"] Jul 26 18:00:34.413107 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.413077 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:00:34.499714 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.499660 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eefed328-c33c-4529-ac0c-92415c3cafd5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq\" (UID: \"eefed328-c33c-4529-ac0c-92415c3cafd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:34.499898 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.499723 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eefed328-c33c-4529-ac0c-92415c3cafd5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq\" (UID: \"eefed328-c33c-4529-ac0c-92415c3cafd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:34.499898 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.499759 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eefed328-c33c-4529-ac0c-92415c3cafd5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq\" (UID: \"eefed328-c33c-4529-ac0c-92415c3cafd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:34.499898 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.499778 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eefed328-c33c-4529-ac0c-92415c3cafd5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq\" (UID: \"eefed328-c33c-4529-ac0c-92415c3cafd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:34.499898 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.499837 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eefed328-c33c-4529-ac0c-92415c3cafd5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq\" (UID: \"eefed328-c33c-4529-ac0c-92415c3cafd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:34.499898 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.499880 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsqgf\" (UniqueName: \"kubernetes.io/projected/eefed328-c33c-4529-ac0c-92415c3cafd5-kube-api-access-hsqgf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq\" (UID: \"eefed328-c33c-4529-ac0c-92415c3cafd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:34.600577 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.600538 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eefed328-c33c-4529-ac0c-92415c3cafd5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq\" (UID: \"eefed328-c33c-4529-ac0c-92415c3cafd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:34.600775 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.600602 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eefed328-c33c-4529-ac0c-92415c3cafd5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq\" (UID: \"eefed328-c33c-4529-ac0c-92415c3cafd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:34.600775 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.600655 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eefed328-c33c-4529-ac0c-92415c3cafd5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq\" (UID: \"eefed328-c33c-4529-ac0c-92415c3cafd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:34.600775 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.600697 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eefed328-c33c-4529-ac0c-92415c3cafd5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq\" (UID: \"eefed328-c33c-4529-ac0c-92415c3cafd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:34.600775 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.600729 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eefed328-c33c-4529-ac0c-92415c3cafd5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq\" (UID: \"eefed328-c33c-4529-ac0c-92415c3cafd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:34.600995 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.600835 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsqgf\" (UniqueName: \"kubernetes.io/projected/eefed328-c33c-4529-ac0c-92415c3cafd5-kube-api-access-hsqgf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq\" (UID: \"eefed328-c33c-4529-ac0c-92415c3cafd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:34.601128 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.601098 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eefed328-c33c-4529-ac0c-92415c3cafd5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq\" (UID: \"eefed328-c33c-4529-ac0c-92415c3cafd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:34.601128 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.601112 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eefed328-c33c-4529-ac0c-92415c3cafd5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq\" (UID: \"eefed328-c33c-4529-ac0c-92415c3cafd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:34.601254 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.601164 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eefed328-c33c-4529-ac0c-92415c3cafd5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq\" (UID: \"eefed328-c33c-4529-ac0c-92415c3cafd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:34.602909 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.602882 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eefed328-c33c-4529-ac0c-92415c3cafd5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq\" (UID: \"eefed328-c33c-4529-ac0c-92415c3cafd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:34.603151 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.603135 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eefed328-c33c-4529-ac0c-92415c3cafd5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq\" (UID: \"eefed328-c33c-4529-ac0c-92415c3cafd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:34.609280 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.609253 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsqgf\" (UniqueName: \"kubernetes.io/projected/eefed328-c33c-4529-ac0c-92415c3cafd5-kube-api-access-hsqgf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq\" (UID: \"eefed328-c33c-4529-ac0c-92415c3cafd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:34.701399 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.701301 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:34.840775 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:34.840739 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq"] Jul 26 18:00:34.842970 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:00:34.842940 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeefed328_c33c_4529_ac0c_92415c3cafd5.slice/crio-495a9dcf067ba964a2a8728ff8f57df903e20661ee388127d82f370e0d8c01f3 WatchSource:0}: Error finding container 495a9dcf067ba964a2a8728ff8f57df903e20661ee388127d82f370e0d8c01f3: Status 404 returned error can't find the container with id 495a9dcf067ba964a2a8728ff8f57df903e20661ee388127d82f370e0d8c01f3 Jul 26 18:00:35.183737 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:35.183708 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" event={"ID":"eefed328-c33c-4529-ac0c-92415c3cafd5","Type":"ContainerStarted","Data":"3f76a66cc786a28386ea9291f9df3eb39f970647f47ce77f78cb98bd7f778a68"} Jul 26 18:00:35.183890 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:35.183759 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" event={"ID":"eefed328-c33c-4529-ac0c-92415c3cafd5","Type":"ContainerStarted","Data":"495a9dcf067ba964a2a8728ff8f57df903e20661ee388127d82f370e0d8c01f3"} Jul 26 18:00:35.202663 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:35.202609 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" podStartSLOduration=0.956025449 podStartE2EDuration="1.202593127s" totalImagesPullingTime="246.567678ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:00:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:00:35.200895097 +0000 UTC m=+689.153550815" watchObservedRunningTime="2026-07-26 18:00:35.202593127 +0000 UTC m=+689.155248814" Jul 26 18:00:41.089194 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.089164 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb"] Jul 26 18:00:41.092659 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.092636 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:41.095464 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.095442 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 26 18:00:41.103770 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.103748 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb"] Jul 26 18:00:41.131112 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.131079 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:00:41.262734 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.262700 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhjhb\" (UID: \"5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:41.262921 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.262750 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhjhb\" (UID: \"5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:41.262921 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.262778 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhjhb\" (UID: \"5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:41.262921 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.262821 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhjhb\" (UID: \"5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:41.262921 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.262847 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhjhb\" (UID: \"5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:41.262921 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.262882 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9785\" (UniqueName: \"kubernetes.io/projected/5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4-kube-api-access-r9785\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhjhb\" (UID: \"5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:41.363955 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.363862 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhjhb\" (UID: \"5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:41.363955 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.363903 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhjhb\" (UID: \"5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:41.363955 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.363929 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9785\" (UniqueName: \"kubernetes.io/projected/5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4-kube-api-access-r9785\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhjhb\" (UID: \"5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:41.364227 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.364047 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhjhb\" (UID: \"5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:41.364227 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.364081 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhjhb\" (UID: \"5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:41.364227 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.364102 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhjhb\" (UID: \"5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:41.364386 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.364319 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhjhb\" (UID: \"5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:41.364462 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.364442 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhjhb\" (UID: \"5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:41.364516 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.364484 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhjhb\" (UID: \"5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:41.366372 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.366340 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhjhb\" (UID: \"5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:41.366600 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.366581 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhjhb\" (UID: \"5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:41.371734 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.371699 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9785\" (UniqueName: \"kubernetes.io/projected/5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4-kube-api-access-r9785\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhjhb\" (UID: \"5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:41.404785 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.404752 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:41.538251 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:41.538227 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb"] Jul 26 18:00:41.540087 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:00:41.540058 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cbc9127_b44e_4a3d_9ebf_8414fa0aacf4.slice/crio-e7c982e15542dad285eb25964eee28a60c2e5feb016c69a0c2f8d4a46bbfb02a WatchSource:0}: Error finding container e7c982e15542dad285eb25964eee28a60c2e5feb016c69a0c2f8d4a46bbfb02a: Status 404 returned error can't find the container with id e7c982e15542dad285eb25964eee28a60c2e5feb016c69a0c2f8d4a46bbfb02a Jul 26 18:00:42.217363 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:42.217330 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" event={"ID":"5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4","Type":"ContainerStarted","Data":"4f2164a283ce880243b70fa9a033c64142576b4a0b8c71f08cfbed4b20055d9b"} Jul 26 18:00:42.217758 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:42.217381 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" event={"ID":"5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4","Type":"ContainerStarted","Data":"e7c982e15542dad285eb25964eee28a60c2e5feb016c69a0c2f8d4a46bbfb02a"} Jul 26 18:00:42.234800 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:42.234749 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" podStartSLOduration=0.97943082 podStartE2EDuration="1.234731438s" totalImagesPullingTime="255.300618ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:00:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:00:42.234055443 +0000 UTC m=+696.186711167" watchObservedRunningTime="2026-07-26 18:00:42.234731438 +0000 UTC m=+696.187387125" Jul 26 18:00:44.701751 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:44.701707 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:44.702080 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:44.701830 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:44.714062 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:44.714038 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:45.241842 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:45.241816 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq" Jul 26 18:00:51.405169 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:51.405127 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:51.405609 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:51.405219 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:51.418185 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:51.418147 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:52.277708 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:52.277662 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhjhb" Jul 26 18:00:53.358904 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:00:53.358849 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:01:49.195622 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:01:49.195577 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:02:14.056409 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:02:14.056370 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:02:14.140766 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:02:14.140733 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:02:15.140093 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:02:15.140059 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:02:15.240028 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:02:15.239985 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:02:26.209402 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:02:26.209363 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:03:01.895558 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:03:01.895524 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67b95f7b6f-c8mtn"] Jul 26 18:03:01.896145 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:03:01.895829 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67b95f7b6f-c8mtn" podUID="f048d6a1-bf88-454f-873e-6f0c97259f06" containerName="manager" containerID="cri-o://b32293cbad2cf82e9ac753df3eae42371588d15d7e1229ad657570cfe9df4bb8" gracePeriod=30 Jul 26 18:03:02.139889 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:03:02.139867 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b95f7b6f-c8mtn" Jul 26 18:03:02.274577 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:03:02.274545 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfrhn\" (UniqueName: \"kubernetes.io/projected/f048d6a1-bf88-454f-873e-6f0c97259f06-kube-api-access-mfrhn\") pod \"f048d6a1-bf88-454f-873e-6f0c97259f06\" (UID: \"f048d6a1-bf88-454f-873e-6f0c97259f06\") " Jul 26 18:03:02.274787 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:03:02.274591 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f048d6a1-bf88-454f-873e-6f0c97259f06-webhook-cert\") pod \"f048d6a1-bf88-454f-873e-6f0c97259f06\" (UID: \"f048d6a1-bf88-454f-873e-6f0c97259f06\") " Jul 26 18:03:02.276647 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:03:02.276618 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f048d6a1-bf88-454f-873e-6f0c97259f06-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f048d6a1-bf88-454f-873e-6f0c97259f06" (UID: "f048d6a1-bf88-454f-873e-6f0c97259f06"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 26 18:03:02.276647 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:03:02.276636 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f048d6a1-bf88-454f-873e-6f0c97259f06-kube-api-access-mfrhn" (OuterVolumeSpecName: "kube-api-access-mfrhn") pod "f048d6a1-bf88-454f-873e-6f0c97259f06" (UID: "f048d6a1-bf88-454f-873e-6f0c97259f06"). InnerVolumeSpecName "kube-api-access-mfrhn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:03:02.376213 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:03:02.376175 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfrhn\" (UniqueName: \"kubernetes.io/projected/f048d6a1-bf88-454f-873e-6f0c97259f06-kube-api-access-mfrhn\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:03:02.376213 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:03:02.376206 2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f048d6a1-bf88-454f-873e-6f0c97259f06-webhook-cert\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:03:02.839617 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:03:02.839582 2559 generic.go:356] "Generic (PLEG): container finished" podID="f048d6a1-bf88-454f-873e-6f0c97259f06" containerID="b32293cbad2cf82e9ac753df3eae42371588d15d7e1229ad657570cfe9df4bb8" exitCode=0 Jul 26 18:03:02.839797 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:03:02.839630 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b95f7b6f-c8mtn" Jul 26 18:03:02.839797 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:03:02.839632 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b95f7b6f-c8mtn" event={"ID":"f048d6a1-bf88-454f-873e-6f0c97259f06","Type":"ContainerDied","Data":"b32293cbad2cf82e9ac753df3eae42371588d15d7e1229ad657570cfe9df4bb8"} Jul 26 18:03:02.839797 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:03:02.839702 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b95f7b6f-c8mtn" event={"ID":"f048d6a1-bf88-454f-873e-6f0c97259f06","Type":"ContainerDied","Data":"0c352f9d096d7f33845978935e47f652ca503d9321f204bf15d38d67771752b0"} Jul 26 18:03:02.839797 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:03:02.839728 2559 scope.go:117] "RemoveContainer" containerID="b32293cbad2cf82e9ac753df3eae42371588d15d7e1229ad657570cfe9df4bb8" Jul 26 18:03:02.848166 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:03:02.848146 2559 scope.go:117] "RemoveContainer" containerID="b32293cbad2cf82e9ac753df3eae42371588d15d7e1229ad657570cfe9df4bb8" Jul 26 18:03:02.848418 ip-10-0-128-209 kubenswrapper[2559]: E0726 18:03:02.848399 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b32293cbad2cf82e9ac753df3eae42371588d15d7e1229ad657570cfe9df4bb8\": container with ID starting with b32293cbad2cf82e9ac753df3eae42371588d15d7e1229ad657570cfe9df4bb8 not found: ID does not exist" containerID="b32293cbad2cf82e9ac753df3eae42371588d15d7e1229ad657570cfe9df4bb8" Jul 26 18:03:02.848489 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:03:02.848430 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b32293cbad2cf82e9ac753df3eae42371588d15d7e1229ad657570cfe9df4bb8"} err="failed to get container status \"b32293cbad2cf82e9ac753df3eae42371588d15d7e1229ad657570cfe9df4bb8\": rpc error: code = NotFound desc = could not find container \"b32293cbad2cf82e9ac753df3eae42371588d15d7e1229ad657570cfe9df4bb8\": container with ID starting with b32293cbad2cf82e9ac753df3eae42371588d15d7e1229ad657570cfe9df4bb8 not found: ID does not exist" Jul 26 18:03:02.860520 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:03:02.860493 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67b95f7b6f-c8mtn"] Jul 26 18:03:02.863719 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:03:02.863699 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67b95f7b6f-c8mtn"] Jul 26 18:03:04.438765 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:03:04.438732 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f048d6a1-bf88-454f-873e-6f0c97259f06" path="/var/lib/kubelet/pods/f048d6a1-bf88-454f-873e-6f0c97259f06/volumes" Jul 26 18:03:38.339346 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:03:38.339308 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:04:03.671828 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:03.671793 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67b95f7b6f-wvpjd"] Jul 26 18:04:03.672211 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:03.672136 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f048d6a1-bf88-454f-873e-6f0c97259f06" containerName="manager" Jul 26 18:04:03.672211 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:03.672146 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f048d6a1-bf88-454f-873e-6f0c97259f06" containerName="manager" Jul 26 18:04:03.672211 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:03.672203 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f048d6a1-bf88-454f-873e-6f0c97259f06" containerName="manager" Jul 26 18:04:03.675036 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:03.675020 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b95f7b6f-wvpjd" Jul 26 18:04:03.678602 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:03.678579 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ccp5l\"" Jul 26 18:04:03.678758 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:03.678625 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 26 18:04:03.678758 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:03.678625 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 26 18:04:03.685861 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:03.685839 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b95f7b6f-wvpjd"] Jul 26 18:04:03.816220 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:03.816184 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9mtn\" (UniqueName: \"kubernetes.io/projected/3fdf2817-4d83-4a4c-8def-b77784a0b90a-kube-api-access-p9mtn\") pod \"maas-controller-67b95f7b6f-wvpjd\" (UID: \"3fdf2817-4d83-4a4c-8def-b77784a0b90a\") " pod="opendatahub/maas-controller-67b95f7b6f-wvpjd" Jul 26 18:04:03.816378 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:03.816229 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3fdf2817-4d83-4a4c-8def-b77784a0b90a-webhook-cert\") pod \"maas-controller-67b95f7b6f-wvpjd\" (UID: \"3fdf2817-4d83-4a4c-8def-b77784a0b90a\") " pod="opendatahub/maas-controller-67b95f7b6f-wvpjd" Jul 26 18:04:03.916888 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:03.916853 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9mtn\" (UniqueName: \"kubernetes.io/projected/3fdf2817-4d83-4a4c-8def-b77784a0b90a-kube-api-access-p9mtn\") pod \"maas-controller-67b95f7b6f-wvpjd\" (UID: \"3fdf2817-4d83-4a4c-8def-b77784a0b90a\") " pod="opendatahub/maas-controller-67b95f7b6f-wvpjd" Jul 26 18:04:03.917086 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:03.916900 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3fdf2817-4d83-4a4c-8def-b77784a0b90a-webhook-cert\") pod \"maas-controller-67b95f7b6f-wvpjd\" (UID: \"3fdf2817-4d83-4a4c-8def-b77784a0b90a\") " pod="opendatahub/maas-controller-67b95f7b6f-wvpjd" Jul 26 18:04:03.919284 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:03.919259 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3fdf2817-4d83-4a4c-8def-b77784a0b90a-webhook-cert\") pod \"maas-controller-67b95f7b6f-wvpjd\" (UID: \"3fdf2817-4d83-4a4c-8def-b77784a0b90a\") " pod="opendatahub/maas-controller-67b95f7b6f-wvpjd" Jul 26 18:04:03.926369 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:03.926316 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9mtn\" (UniqueName: \"kubernetes.io/projected/3fdf2817-4d83-4a4c-8def-b77784a0b90a-kube-api-access-p9mtn\") pod \"maas-controller-67b95f7b6f-wvpjd\" (UID: \"3fdf2817-4d83-4a4c-8def-b77784a0b90a\") " pod="opendatahub/maas-controller-67b95f7b6f-wvpjd" Jul 26 18:04:03.985311 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:03.985278 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b95f7b6f-wvpjd" Jul 26 18:04:04.117340 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:04.117309 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b95f7b6f-wvpjd"] Jul 26 18:04:04.119475 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:04:04.119437 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fdf2817_4d83_4a4c_8def_b77784a0b90a.slice/crio-09c7f47763e7739005e15a181b27af7a30da58fcf1f090b527543fe49b5088d9 WatchSource:0}: Error finding container 09c7f47763e7739005e15a181b27af7a30da58fcf1f090b527543fe49b5088d9: Status 404 returned error can't find the container with id 09c7f47763e7739005e15a181b27af7a30da58fcf1f090b527543fe49b5088d9 Jul 26 18:04:05.103718 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:05.103681 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b95f7b6f-wvpjd" event={"ID":"3fdf2817-4d83-4a4c-8def-b77784a0b90a","Type":"ContainerStarted","Data":"7702826ca2aea68f18f0d344b8812c97c10a48d4b854ec1bf18461130bea0ebb"} Jul 26 18:04:05.104137 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:05.103736 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b95f7b6f-wvpjd" event={"ID":"3fdf2817-4d83-4a4c-8def-b77784a0b90a","Type":"ContainerStarted","Data":"09c7f47763e7739005e15a181b27af7a30da58fcf1f090b527543fe49b5088d9"} Jul 26 18:04:05.104137 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:05.103789 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67b95f7b6f-wvpjd" Jul 26 18:04:05.123958 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:05.123911 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67b95f7b6f-wvpjd" podStartSLOduration=1.6456856069999999 podStartE2EDuration="2.123895336s" totalImagesPullingTime="478.209729ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:04:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:04:05.122022839 +0000 UTC m=+899.074678552" watchObservedRunningTime="2026-07-26 18:04:05.123895336 +0000 UTC m=+899.076551023" Jul 26 18:04:16.110986 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:16.110952 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67b95f7b6f-wvpjd" Jul 26 18:04:20.665449 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:20.665418 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:04:20.769075 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:20.769042 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67b95f7b6f-wvpjd"] Jul 26 18:04:20.769274 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:20.769232 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67b95f7b6f-wvpjd" podUID="3fdf2817-4d83-4a4c-8def-b77784a0b90a" containerName="manager" containerID="cri-o://7702826ca2aea68f18f0d344b8812c97c10a48d4b854ec1bf18461130bea0ebb" gracePeriod=30 Jul 26 18:04:21.016087 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:21.016061 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b95f7b6f-wvpjd" Jul 26 18:04:21.063452 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:21.063421 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9mtn\" (UniqueName: \"kubernetes.io/projected/3fdf2817-4d83-4a4c-8def-b77784a0b90a-kube-api-access-p9mtn\") pod \"3fdf2817-4d83-4a4c-8def-b77784a0b90a\" (UID: \"3fdf2817-4d83-4a4c-8def-b77784a0b90a\") " Jul 26 18:04:21.063641 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:21.063509 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3fdf2817-4d83-4a4c-8def-b77784a0b90a-webhook-cert\") pod \"3fdf2817-4d83-4a4c-8def-b77784a0b90a\" (UID: \"3fdf2817-4d83-4a4c-8def-b77784a0b90a\") " Jul 26 18:04:21.065495 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:21.065456 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fdf2817-4d83-4a4c-8def-b77784a0b90a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3fdf2817-4d83-4a4c-8def-b77784a0b90a" (UID: "3fdf2817-4d83-4a4c-8def-b77784a0b90a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 26 18:04:21.065606 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:21.065558 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fdf2817-4d83-4a4c-8def-b77784a0b90a-kube-api-access-p9mtn" (OuterVolumeSpecName: "kube-api-access-p9mtn") pod "3fdf2817-4d83-4a4c-8def-b77784a0b90a" (UID: "3fdf2817-4d83-4a4c-8def-b77784a0b90a"). InnerVolumeSpecName "kube-api-access-p9mtn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:04:21.164324 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:21.164280 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9mtn\" (UniqueName: \"kubernetes.io/projected/3fdf2817-4d83-4a4c-8def-b77784a0b90a-kube-api-access-p9mtn\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:04:21.164324 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:21.164317 2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3fdf2817-4d83-4a4c-8def-b77784a0b90a-webhook-cert\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:04:21.174925 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:21.174888 2559 generic.go:356] "Generic (PLEG): container finished" podID="3fdf2817-4d83-4a4c-8def-b77784a0b90a" containerID="7702826ca2aea68f18f0d344b8812c97c10a48d4b854ec1bf18461130bea0ebb" exitCode=0 Jul 26 18:04:21.175063 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:21.174937 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b95f7b6f-wvpjd" Jul 26 18:04:21.175063 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:21.174933 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b95f7b6f-wvpjd" event={"ID":"3fdf2817-4d83-4a4c-8def-b77784a0b90a","Type":"ContainerDied","Data":"7702826ca2aea68f18f0d344b8812c97c10a48d4b854ec1bf18461130bea0ebb"} Jul 26 18:04:21.175063 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:21.174994 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b95f7b6f-wvpjd" event={"ID":"3fdf2817-4d83-4a4c-8def-b77784a0b90a","Type":"ContainerDied","Data":"09c7f47763e7739005e15a181b27af7a30da58fcf1f090b527543fe49b5088d9"} Jul 26 18:04:21.175063 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:21.175021 2559 scope.go:117] "RemoveContainer" containerID="7702826ca2aea68f18f0d344b8812c97c10a48d4b854ec1bf18461130bea0ebb" Jul 26 18:04:21.184325 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:21.184304 2559 scope.go:117] "RemoveContainer" containerID="7702826ca2aea68f18f0d344b8812c97c10a48d4b854ec1bf18461130bea0ebb" Jul 26 18:04:21.184601 ip-10-0-128-209 kubenswrapper[2559]: E0726 18:04:21.184580 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7702826ca2aea68f18f0d344b8812c97c10a48d4b854ec1bf18461130bea0ebb\": container with ID starting with 7702826ca2aea68f18f0d344b8812c97c10a48d4b854ec1bf18461130bea0ebb not found: ID does not exist" containerID="7702826ca2aea68f18f0d344b8812c97c10a48d4b854ec1bf18461130bea0ebb" Jul 26 18:04:21.184658 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:21.184612 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7702826ca2aea68f18f0d344b8812c97c10a48d4b854ec1bf18461130bea0ebb"} err="failed to get container status \"7702826ca2aea68f18f0d344b8812c97c10a48d4b854ec1bf18461130bea0ebb\": rpc error: code = NotFound desc = could not find container \"7702826ca2aea68f18f0d344b8812c97c10a48d4b854ec1bf18461130bea0ebb\": container with ID starting with 7702826ca2aea68f18f0d344b8812c97c10a48d4b854ec1bf18461130bea0ebb not found: ID does not exist" Jul 26 18:04:21.197461 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:21.197405 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67b95f7b6f-wvpjd"] Jul 26 18:04:21.201360 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:21.201332 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67b95f7b6f-wvpjd"] Jul 26 18:04:21.921049 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:21.921003 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:04:22.030446 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:22.030407 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67b95f7b6f-krcq4"] Jul 26 18:04:22.030760 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:22.030748 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fdf2817-4d83-4a4c-8def-b77784a0b90a" containerName="manager" Jul 26 18:04:22.030811 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:22.030761 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fdf2817-4d83-4a4c-8def-b77784a0b90a" containerName="manager" Jul 26 18:04:22.030847 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:22.030843 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fdf2817-4d83-4a4c-8def-b77784a0b90a" containerName="manager" Jul 26 18:04:22.034969 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:22.034949 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b95f7b6f-krcq4" Jul 26 18:04:22.037584 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:22.037557 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 26 18:04:22.038564 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:22.038545 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 26 18:04:22.038659 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:22.038597 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ccp5l\"" Jul 26 18:04:22.051345 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:22.051324 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b95f7b6f-krcq4"] Jul 26 18:04:22.070753 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:22.070725 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n62hs\" (UniqueName: \"kubernetes.io/projected/2098ad1a-2857-4700-ab98-922266c839f5-kube-api-access-n62hs\") pod \"maas-controller-67b95f7b6f-krcq4\" (UID: \"2098ad1a-2857-4700-ab98-922266c839f5\") " pod="opendatahub/maas-controller-67b95f7b6f-krcq4" Jul 26 18:04:22.070871 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:22.070763 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2098ad1a-2857-4700-ab98-922266c839f5-webhook-cert\") pod \"maas-controller-67b95f7b6f-krcq4\" (UID: \"2098ad1a-2857-4700-ab98-922266c839f5\") " pod="opendatahub/maas-controller-67b95f7b6f-krcq4" Jul 26 18:04:22.171607 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:22.171518 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2098ad1a-2857-4700-ab98-922266c839f5-webhook-cert\") pod \"maas-controller-67b95f7b6f-krcq4\" (UID: \"2098ad1a-2857-4700-ab98-922266c839f5\") " pod="opendatahub/maas-controller-67b95f7b6f-krcq4" Jul 26 18:04:22.171810 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:22.171612 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n62hs\" (UniqueName: \"kubernetes.io/projected/2098ad1a-2857-4700-ab98-922266c839f5-kube-api-access-n62hs\") pod \"maas-controller-67b95f7b6f-krcq4\" (UID: \"2098ad1a-2857-4700-ab98-922266c839f5\") " pod="opendatahub/maas-controller-67b95f7b6f-krcq4" Jul 26 18:04:22.173994 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:22.173969 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2098ad1a-2857-4700-ab98-922266c839f5-webhook-cert\") pod \"maas-controller-67b95f7b6f-krcq4\" (UID: \"2098ad1a-2857-4700-ab98-922266c839f5\") " pod="opendatahub/maas-controller-67b95f7b6f-krcq4" Jul 26 18:04:22.181207 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:22.181184 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n62hs\" (UniqueName: \"kubernetes.io/projected/2098ad1a-2857-4700-ab98-922266c839f5-kube-api-access-n62hs\") pod \"maas-controller-67b95f7b6f-krcq4\" (UID: \"2098ad1a-2857-4700-ab98-922266c839f5\") " pod="opendatahub/maas-controller-67b95f7b6f-krcq4" Jul 26 18:04:22.344964 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:22.344927 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67b95f7b6f-krcq4" Jul 26 18:04:22.438854 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:22.438824 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fdf2817-4d83-4a4c-8def-b77784a0b90a" path="/var/lib/kubelet/pods/3fdf2817-4d83-4a4c-8def-b77784a0b90a/volumes" Jul 26 18:04:22.477453 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:22.477418 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67b95f7b6f-krcq4"] Jul 26 18:04:22.478255 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:04:22.478229 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2098ad1a_2857_4700_ab98_922266c839f5.slice/crio-5f7d42ae86e25b8ed2e8671a6969547db65f8b5c9dd6c06a8c19df2375c6adec WatchSource:0}: Error finding container 5f7d42ae86e25b8ed2e8671a6969547db65f8b5c9dd6c06a8c19df2375c6adec: Status 404 returned error can't find the container with id 5f7d42ae86e25b8ed2e8671a6969547db65f8b5c9dd6c06a8c19df2375c6adec Jul 26 18:04:23.186954 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:23.186870 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b95f7b6f-krcq4" event={"ID":"2098ad1a-2857-4700-ab98-922266c839f5","Type":"ContainerStarted","Data":"c7a7134182e720a1f03659e52560df65e16d1b887d941b9af0d114d8d15d952a"} Jul 26 18:04:23.186954 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:23.186922 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67b95f7b6f-krcq4" event={"ID":"2098ad1a-2857-4700-ab98-922266c839f5","Type":"ContainerStarted","Data":"5f7d42ae86e25b8ed2e8671a6969547db65f8b5c9dd6c06a8c19df2375c6adec"} Jul 26 18:04:23.186954 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:23.186946 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67b95f7b6f-krcq4" Jul 26 18:04:23.207717 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:23.207642 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67b95f7b6f-krcq4" podStartSLOduration=0.836061618 podStartE2EDuration="1.207620455s" totalImagesPullingTime="371.558837ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:04:23.205055757 +0000 UTC m=+917.157711455" watchObservedRunningTime="2026-07-26 18:04:23.207620455 +0000 UTC m=+917.160276146" Jul 26 18:04:34.193371 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:34.193341 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67b95f7b6f-krcq4" Jul 26 18:04:43.235787 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:43.235752 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:04:59.513496 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:59.513450 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:04:59.615369 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:59.615332 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:04:59.813395 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:04:59.813273 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:05:06.192021 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:05:06.191971 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:05:06.335833 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:05:06.335794 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:05:06.413885 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:05:06.413857 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:05:06.539822 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:05:06.539785 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:05:12.245022 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:05:12.244970 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:05:12.316985 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:05:12.316945 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:07:09.628777 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:07:09.628699 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:08:39.440451 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:08:39.440363 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:09:07.504774 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:09:07.504739 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76"] Jul 26 18:09:07.505153 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:09:07.504938 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76" podUID="0d262968-3507-422b-977f-ccf2d69fedfd" containerName="manager" containerID="cri-o://758a41aee07fbe02bcd79a7146a9f7e448f1114b62d2442c4c867e5817db5618" gracePeriod=10 Jul 26 18:09:10.742941 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:09:10.742915 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76" Jul 26 18:09:10.826201 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:09:10.826105 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d262968-3507-422b-977f-ccf2d69fedfd-extensions-socket-volume\") pod \"0d262968-3507-422b-977f-ccf2d69fedfd\" (UID: \"0d262968-3507-422b-977f-ccf2d69fedfd\") " Jul 26 18:09:10.826201 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:09:10.826168 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmnjt\" (UniqueName: \"kubernetes.io/projected/0d262968-3507-422b-977f-ccf2d69fedfd-kube-api-access-rmnjt\") pod \"0d262968-3507-422b-977f-ccf2d69fedfd\" (UID: \"0d262968-3507-422b-977f-ccf2d69fedfd\") " Jul 26 18:09:10.826528 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:09:10.826504 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d262968-3507-422b-977f-ccf2d69fedfd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0d262968-3507-422b-977f-ccf2d69fedfd" (UID: "0d262968-3507-422b-977f-ccf2d69fedfd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:09:10.828173 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:09:10.828149 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d262968-3507-422b-977f-ccf2d69fedfd-kube-api-access-rmnjt" (OuterVolumeSpecName: "kube-api-access-rmnjt") pod "0d262968-3507-422b-977f-ccf2d69fedfd" (UID: "0d262968-3507-422b-977f-ccf2d69fedfd"). InnerVolumeSpecName "kube-api-access-rmnjt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:09:10.927777 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:09:10.927736 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d262968-3507-422b-977f-ccf2d69fedfd-extensions-socket-volume\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:09:10.927777 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:09:10.927769 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmnjt\" (UniqueName: \"kubernetes.io/projected/0d262968-3507-422b-977f-ccf2d69fedfd-kube-api-access-rmnjt\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:09:11.437204 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:09:11.437171 2559 generic.go:356] "Generic (PLEG): container finished" podID="0d262968-3507-422b-977f-ccf2d69fedfd" containerID="758a41aee07fbe02bcd79a7146a9f7e448f1114b62d2442c4c867e5817db5618" exitCode=0 Jul 26 18:09:11.437383 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:09:11.437216 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76" Jul 26 18:09:11.437383 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:09:11.437230 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76" event={"ID":"0d262968-3507-422b-977f-ccf2d69fedfd","Type":"ContainerDied","Data":"758a41aee07fbe02bcd79a7146a9f7e448f1114b62d2442c4c867e5817db5618"} Jul 26 18:09:11.437383 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:09:11.437269 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76" event={"ID":"0d262968-3507-422b-977f-ccf2d69fedfd","Type":"ContainerDied","Data":"a325788ce5df17292a5129717fb0b4adfee20ed5268ed26cac4fa73a8ba24df7"} Jul 26 18:09:11.437383 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:09:11.437289 2559 scope.go:117] "RemoveContainer" containerID="758a41aee07fbe02bcd79a7146a9f7e448f1114b62d2442c4c867e5817db5618" Jul 26 18:09:11.447281 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:09:11.447263 2559 scope.go:117] "RemoveContainer" containerID="758a41aee07fbe02bcd79a7146a9f7e448f1114b62d2442c4c867e5817db5618" Jul 26 18:09:11.447541 ip-10-0-128-209 kubenswrapper[2559]: E0726 18:09:11.447525 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"758a41aee07fbe02bcd79a7146a9f7e448f1114b62d2442c4c867e5817db5618\": container with ID starting with 758a41aee07fbe02bcd79a7146a9f7e448f1114b62d2442c4c867e5817db5618 not found: ID does not exist" containerID="758a41aee07fbe02bcd79a7146a9f7e448f1114b62d2442c4c867e5817db5618" Jul 26 18:09:11.447601 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:09:11.447548 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"758a41aee07fbe02bcd79a7146a9f7e448f1114b62d2442c4c867e5817db5618"} err="failed to get container status \"758a41aee07fbe02bcd79a7146a9f7e448f1114b62d2442c4c867e5817db5618\": rpc error: code = NotFound desc = could not find container \"758a41aee07fbe02bcd79a7146a9f7e448f1114b62d2442c4c867e5817db5618\": container with ID starting with 758a41aee07fbe02bcd79a7146a9f7e448f1114b62d2442c4c867e5817db5618 not found: ID does not exist" Jul 26 18:09:11.458352 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:09:11.458325 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76"] Jul 26 18:09:11.461529 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:09:11.461507 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tnf76"] Jul 26 18:09:12.438992 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:09:12.438952 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d262968-3507-422b-977f-ccf2d69fedfd" path="/var/lib/kubelet/pods/0d262968-3507-422b-977f-ccf2d69fedfd/volumes" Jul 26 18:10:08.649379 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:10:08.649344 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nxqxj"] Jul 26 18:10:08.649788 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:10:08.649697 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d262968-3507-422b-977f-ccf2d69fedfd" containerName="manager" Jul 26 18:10:08.649788 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:10:08.649712 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d262968-3507-422b-977f-ccf2d69fedfd" containerName="manager" Jul 26 18:10:08.649788 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:10:08.649784 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d262968-3507-422b-977f-ccf2d69fedfd" containerName="manager" Jul 26 18:10:08.653841 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:10:08.653824 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nxqxj" Jul 26 18:10:08.656476 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:10:08.656452 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rqdcd\"" Jul 26 18:10:08.664622 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:10:08.664603 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nxqxj"] Jul 26 18:10:08.723247 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:10:08.723210 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gclr\" (UniqueName: \"kubernetes.io/projected/15d2d1c7-390b-450b-82e9-2d5b49a022f1-kube-api-access-4gclr\") pod \"kuadrant-operator-controller-manager-86878c769f-nxqxj\" (UID: \"15d2d1c7-390b-450b-82e9-2d5b49a022f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nxqxj" Jul 26 18:10:08.723434 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:10:08.723337 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15d2d1c7-390b-450b-82e9-2d5b49a022f1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nxqxj\" (UID: \"15d2d1c7-390b-450b-82e9-2d5b49a022f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nxqxj" Jul 26 18:10:08.824336 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:10:08.824297 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15d2d1c7-390b-450b-82e9-2d5b49a022f1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nxqxj\" (UID: \"15d2d1c7-390b-450b-82e9-2d5b49a022f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nxqxj" Jul 26 18:10:08.824486 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:10:08.824348 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gclr\" (UniqueName: \"kubernetes.io/projected/15d2d1c7-390b-450b-82e9-2d5b49a022f1-kube-api-access-4gclr\") pod \"kuadrant-operator-controller-manager-86878c769f-nxqxj\" (UID: \"15d2d1c7-390b-450b-82e9-2d5b49a022f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nxqxj" Jul 26 18:10:08.824716 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:10:08.824699 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15d2d1c7-390b-450b-82e9-2d5b49a022f1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nxqxj\" (UID: \"15d2d1c7-390b-450b-82e9-2d5b49a022f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nxqxj" Jul 26 18:10:08.832252 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:10:08.832227 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gclr\" (UniqueName: \"kubernetes.io/projected/15d2d1c7-390b-450b-82e9-2d5b49a022f1-kube-api-access-4gclr\") pod \"kuadrant-operator-controller-manager-86878c769f-nxqxj\" (UID: \"15d2d1c7-390b-450b-82e9-2d5b49a022f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nxqxj" Jul 26 18:10:08.964226 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:10:08.964199 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nxqxj" Jul 26 18:10:09.096202 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:10:09.096175 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nxqxj"] Jul 26 18:10:09.098475 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:10:09.098446 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15d2d1c7_390b_450b_82e9_2d5b49a022f1.slice/crio-13275874e5297516684ba10ec62a895b7a2923c039b721d98e874756521c2757 WatchSource:0}: Error finding container 13275874e5297516684ba10ec62a895b7a2923c039b721d98e874756521c2757: Status 404 returned error can't find the container with id 13275874e5297516684ba10ec62a895b7a2923c039b721d98e874756521c2757 Jul 26 18:10:09.678664 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:10:09.678631 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nxqxj" event={"ID":"15d2d1c7-390b-450b-82e9-2d5b49a022f1","Type":"ContainerStarted","Data":"607269f0a0817fd8f3d52f87bae047a04ca5ab78113f37dd109b3acb3488bdca"} Jul 26 18:10:09.679099 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:10:09.678713 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nxqxj" event={"ID":"15d2d1c7-390b-450b-82e9-2d5b49a022f1","Type":"ContainerStarted","Data":"13275874e5297516684ba10ec62a895b7a2923c039b721d98e874756521c2757"} Jul 26 18:10:09.679099 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:10:09.678737 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nxqxj" Jul 26 18:10:09.700207 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:10:09.700153 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nxqxj" podStartSLOduration=1.7001376179999999 podStartE2EDuration="1.700137618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:10:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:10:09.697622601 +0000 UTC m=+1263.650278309" watchObservedRunningTime="2026-07-26 18:10:09.700137618 +0000 UTC m=+1263.652793305" Jul 26 18:10:20.685997 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:10:20.685966 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nxqxj" Jul 26 18:11:53.875446 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:11:53.875408 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:12:52.540985 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:12:52.540953 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:12:52.628322 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:12:52.628287 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:13:05.823956 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:13:05.823921 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:13:45.119368 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:13:45.119330 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:13:46.038348 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:13:46.038312 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:16:13.851647 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:16:13.851574 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:16:13.934201 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:16:13.934166 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:16:51.344927 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:16:51.344884 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:17:12.829972 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:17:12.829935 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:17:16.003743 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:17:16.003711 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:17:16.104264 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:17:16.104213 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:17:16.435692 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:17:16.435570 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:17:22.149547 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:17:22.149515 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:17:22.280796 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:17:22.280767 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:17:22.409206 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:17:22.409124 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:17:22.704381 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:17:22.704303 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:17:23.454846 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:17:23.454790 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:17:25.005176 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:17:25.005136 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:17:25.083576 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:17:25.083535 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:17:25.147929 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:17:25.147893 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:17:25.402180 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:17:25.402087 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:17:25.590053 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:17:25.590017 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:17:25.677105 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:17:25.677016 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:01.101627 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:01.101595 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:01.202263 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:01.202228 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:01.305186 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:01.305158 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:05.603273 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:05.603233 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:05.747603 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:05.747572 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:06.037680 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:06.037621 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:06.228793 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:06.228757 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:06.313336 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:06.313246 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:06.415128 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:06.415087 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:06.744195 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:06.744167 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:06.843920 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:06.843886 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:09.396754 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:09.396718 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:09.500974 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:09.500945 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:10.008336 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:10.008298 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:10.104414 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:10.104384 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:10.202971 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:10.202941 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:43.278951 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:43.278917 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:44.833273 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:44.833235 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:44.906982 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:44.906947 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:44.967534 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:44.967504 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:49.447172 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:49.447132 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:49.618097 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:49.618063 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:49.854756 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:49.854723 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:50.077451 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:50.077422 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:50.381003 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:50.380970 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:50.521387 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:50.521357 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:52.601777 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:52.601746 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:52.698266 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:52.698234 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:52.860643 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:52.860556 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:53.158407 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:53.158321 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:53.300259 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:53.300225 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:18:53.399574 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:18:53.399535 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:19:28.698933 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:19:28.698900 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:19:28.771013 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:19:28.770980 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:19:28.833755 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:19:28.833703 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:19:33.390393 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:19:33.390362 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:19:34.676495 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:19:34.676450 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:19:34.892950 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:19:34.892916 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:19:35.096413 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:19:35.096381 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:19:35.454267 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:19:35.454170 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:19:35.855685 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:19:35.855635 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:19:37.800501 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:19:37.800461 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:19:37.935581 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:19:37.935546 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:19:38.063349 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:19:38.063266 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:19:38.491905 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:19:38.491876 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:19:38.592138 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:19:38.592106 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:19:38.691594 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:19:38.691559 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:20:13.193994 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:20:13.193960 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:20:13.265900 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:20:13.265866 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:20:13.331016 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:20:13.330980 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:20:17.700072 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:20:17.700029 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:20:17.859827 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:20:17.859789 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:20:18.094250 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:20:18.094201 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:20:18.554446 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:20:18.554403 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:20:18.652715 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:20:18.652685 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:20:18.753043 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:20:18.753006 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:20:20.141191 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:20:20.141155 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:20:26.309235 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:20:26.309196 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:20:26.422281 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:20:26.422245 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:20:26.508972 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:20:26.508937 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:20:26.610335 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:20:26.610245 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:20:27.046558 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:20:27.046526 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:20:52.969782 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:20:52.969738 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:20:58.713853 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:20:58.713817 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:20:58.826934 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:20:58.826899 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:20:58.919971 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:20:58.919936 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:14.821732 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:14.821696 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:20.813386 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:20.813352 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:20.875741 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:20.875710 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:20.961623 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:20.961579 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:21.055194 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:21.055156 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:31.548004 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:31.547929 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:32.295684 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:32.295646 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:38.672940 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:38.672904 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:38.777179 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:38.777141 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:38.872127 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:38.872062 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:38.972101 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:38.972070 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:39.712530 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:39.712494 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:39.816897 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:39.816867 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:39.911253 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:39.911217 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:41.459767 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:41.459731 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:47.760096 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:47.760056 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:47.857940 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:47.857909 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:48.450000 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:48.449968 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:54.144819 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:54.144784 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:54.207516 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:54.207480 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:54.389340 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:54.389303 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:54.706640 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:54.706606 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:54.805378 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:54.805348 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:58.193882 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:58.193786 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:58.276117 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:58.276087 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:58.346219 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:58.346183 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:58.490376 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:58.490334 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:58.860232 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:58.860136 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:21:58.955832 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:21:58.955800 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:02.596005 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:02.595966 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:03.239842 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:03.239807 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:03.337893 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:03.337864 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:03.642212 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:03.642132 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:09.458010 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:09.457970 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:09.523398 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:09.523369 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:09.798119 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:09.798040 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d700fe05-lg8m7"] Jul 26 18:22:09.801224 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:09.801207 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d700fe05-lg8m7" Jul 26 18:22:09.803873 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:09.803846 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-swzk5\"" Jul 26 18:22:09.808835 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:09.808503 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d700fe05-lg8m7"] Jul 26 18:22:09.909081 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:09.909031 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ac74ef7-6b83-45aa-a461-78e6da1cd44a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d700fe05-lg8m7\" (UID: \"0ac74ef7-6b83-45aa-a461-78e6da1cd44a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d700fe05-lg8m7" Jul 26 18:22:10.010622 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:10.010590 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ac74ef7-6b83-45aa-a461-78e6da1cd44a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d700fe05-lg8m7\" (UID: \"0ac74ef7-6b83-45aa-a461-78e6da1cd44a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d700fe05-lg8m7" Jul 26 18:22:10.011254 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:10.011233 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ac74ef7-6b83-45aa-a461-78e6da1cd44a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d700fe05-lg8m7\" (UID: \"0ac74ef7-6b83-45aa-a461-78e6da1cd44a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d700fe05-lg8m7" Jul 26 18:22:10.112363 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:10.112280 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d700fe05-lg8m7" Jul 26 18:22:10.251155 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:10.251042 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d700fe05-lg8m7"] Jul 26 18:22:10.253603 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:22:10.253536 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ac74ef7_6b83_45aa_a461_78e6da1cd44a.slice/crio-bd7d687778348219226c23d4cbb35799ba926edf5f7aca4c51fa32334919ccdb WatchSource:0}: Error finding container bd7d687778348219226c23d4cbb35799ba926edf5f7aca4c51fa32334919ccdb: Status 404 returned error can't find the container with id bd7d687778348219226c23d4cbb35799ba926edf5f7aca4c51fa32334919ccdb Jul 26 18:22:10.255656 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:10.255638 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 26 18:22:10.840628 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:10.840593 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d700fe05-lg8m7" event={"ID":"0ac74ef7-6b83-45aa-a461-78e6da1cd44a","Type":"ContainerStarted","Data":"bd7d687778348219226c23d4cbb35799ba926edf5f7aca4c51fa32334919ccdb"} Jul 26 18:22:12.851473 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:12.851437 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d700fe05-lg8m7" event={"ID":"0ac74ef7-6b83-45aa-a461-78e6da1cd44a","Type":"ContainerStarted","Data":"2dedfb97a98c627ac2f7d28a78aa22b611df48b957659999abdd4a59b7b696b5"} Jul 26 18:22:12.867579 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:12.867532 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d700fe05-lg8m7" podStartSLOduration=1.7196080870000001 podStartE2EDuration="3.867515134s" totalImagesPullingTime="2.147907047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:22:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:22:12.865900932 +0000 UTC m=+1986.818556619" watchObservedRunningTime="2026-07-26 18:22:12.867515134 +0000 UTC m=+1986.820170823" Jul 26 18:22:13.856757 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:13.856717 2559 generic.go:356] "Generic (PLEG): container finished" podID="0ac74ef7-6b83-45aa-a461-78e6da1cd44a" containerID="2dedfb97a98c627ac2f7d28a78aa22b611df48b957659999abdd4a59b7b696b5" exitCode=0 Jul 26 18:22:13.857125 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:13.856809 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d700fe05-lg8m7" event={"ID":"0ac74ef7-6b83-45aa-a461-78e6da1cd44a","Type":"ContainerDied","Data":"2dedfb97a98c627ac2f7d28a78aa22b611df48b957659999abdd4a59b7b696b5"} Jul 26 18:22:14.991163 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:14.991136 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d700fe05-lg8m7" Jul 26 18:22:15.157102 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:15.157000 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ac74ef7-6b83-45aa-a461-78e6da1cd44a-maas-api-service-ca\") pod \"0ac74ef7-6b83-45aa-a461-78e6da1cd44a\" (UID: \"0ac74ef7-6b83-45aa-a461-78e6da1cd44a\") " Jul 26 18:22:15.157420 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:15.157400 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ac74ef7-6b83-45aa-a461-78e6da1cd44a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0ac74ef7-6b83-45aa-a461-78e6da1cd44a" (UID: "0ac74ef7-6b83-45aa-a461-78e6da1cd44a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:22:15.257707 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:15.257662 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ac74ef7-6b83-45aa-a461-78e6da1cd44a-maas-api-service-ca\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:22:15.866887 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:15.866850 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d700fe05-lg8m7" Jul 26 18:22:15.866887 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:15.866873 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d700fe05-lg8m7" event={"ID":"0ac74ef7-6b83-45aa-a461-78e6da1cd44a","Type":"ContainerDied","Data":"bd7d687778348219226c23d4cbb35799ba926edf5f7aca4c51fa32334919ccdb"} Jul 26 18:22:15.867127 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:15.866925 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd7d687778348219226c23d4cbb35799ba926edf5f7aca4c51fa32334919ccdb" Jul 26 18:22:20.069994 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:20.069963 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:21.382935 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:21.382903 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:24.871699 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:24.871643 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d700fe05-lg8m7"] Jul 26 18:22:24.874078 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:24.874049 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d700fe05-lg8m7"] Jul 26 18:22:24.875839 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:24.875816 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:24.940826 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:24.940787 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:25.247438 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:25.247410 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:26.438787 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:26.438749 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ac74ef7-6b83-45aa-a461-78e6da1cd44a" path="/var/lib/kubelet/pods/0ac74ef7-6b83-45aa-a461-78e6da1cd44a/volumes" Jul 26 18:22:31.006868 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:31.006827 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:31.107455 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:31.107423 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:31.345492 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:31.345413 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:31.517481 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:31.517445 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:31.701305 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:31.701229 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:31.795526 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:31.795492 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:31.890822 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:31.890788 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:35.033016 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:35.032982 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:37.823883 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:37.823849 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:37.933325 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:37.933281 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:38.024010 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:38.023977 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:38.179837 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:38.179753 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:44.014169 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:44.014133 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:44.115493 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:44.115459 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:50.827400 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:50.827365 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:51.146142 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:51.146066 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:51.248735 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:51.248701 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:54.403257 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:54.403222 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:54.508782 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:54.508749 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:54.606695 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:54.606650 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:54.756835 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:54.756798 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:55.175808 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:55.175727 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:55.291885 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:55.291845 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:22:58.878198 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:22:58.878143 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:23:06.977333 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:06.977298 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:23:28.005005 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:28.004922 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-614abb29-xwh8l"] Jul 26 18:23:28.005442 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:28.005286 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ac74ef7-6b83-45aa-a461-78e6da1cd44a" containerName="revoke-keys" Jul 26 18:23:28.005442 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:28.005298 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ac74ef7-6b83-45aa-a461-78e6da1cd44a" containerName="revoke-keys" Jul 26 18:23:28.005442 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:28.005371 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ac74ef7-6b83-45aa-a461-78e6da1cd44a" containerName="revoke-keys" Jul 26 18:23:28.008443 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:28.008425 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-614abb29-xwh8l" Jul 26 18:23:28.010957 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:28.010934 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-swzk5\"" Jul 26 18:23:28.014794 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:28.014773 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-614abb29-xwh8l"] Jul 26 18:23:28.092700 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:28.092652 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1b42278-92cd-46ca-ac35-c73d52dbc28e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-614abb29-xwh8l\" (UID: \"e1b42278-92cd-46ca-ac35-c73d52dbc28e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-614abb29-xwh8l" Jul 26 18:23:28.193545 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:28.193515 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1b42278-92cd-46ca-ac35-c73d52dbc28e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-614abb29-xwh8l\" (UID: \"e1b42278-92cd-46ca-ac35-c73d52dbc28e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-614abb29-xwh8l" Jul 26 18:23:28.194279 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:28.194257 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1b42278-92cd-46ca-ac35-c73d52dbc28e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-614abb29-xwh8l\" (UID: \"e1b42278-92cd-46ca-ac35-c73d52dbc28e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-614abb29-xwh8l" Jul 26 18:23:28.321494 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:28.321401 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-614abb29-xwh8l" Jul 26 18:23:28.455432 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:28.455406 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-614abb29-xwh8l"] Jul 26 18:23:28.457467 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:23:28.457443 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1b42278_92cd_46ca_ac35_c73d52dbc28e.slice/crio-21151811e558b77ec8977a57ab36becbf24708f1f220687239110c3462ac0319 WatchSource:0}: Error finding container 21151811e558b77ec8977a57ab36becbf24708f1f220687239110c3462ac0319: Status 404 returned error can't find the container with id 21151811e558b77ec8977a57ab36becbf24708f1f220687239110c3462ac0319 Jul 26 18:23:29.185695 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:29.185637 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-614abb29-xwh8l" event={"ID":"e1b42278-92cd-46ca-ac35-c73d52dbc28e","Type":"ContainerStarted","Data":"7688abeab8da0bbef29c75baa32fe7957c774158d743708bdbe7f993924691c8"} Jul 26 18:23:29.185695 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:29.185706 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-614abb29-xwh8l" event={"ID":"e1b42278-92cd-46ca-ac35-c73d52dbc28e","Type":"ContainerStarted","Data":"21151811e558b77ec8977a57ab36becbf24708f1f220687239110c3462ac0319"} Jul 26 18:23:29.201422 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:29.201369 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-614abb29-xwh8l" podStartSLOduration=2.201354863 podStartE2EDuration="2.201354863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:23:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:23:29.199130656 +0000 UTC m=+2063.151786343" watchObservedRunningTime="2026-07-26 18:23:29.201354863 +0000 UTC m=+2063.154010602" Jul 26 18:23:30.190978 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:30.190945 2559 generic.go:356] "Generic (PLEG): container finished" podID="e1b42278-92cd-46ca-ac35-c73d52dbc28e" containerID="7688abeab8da0bbef29c75baa32fe7957c774158d743708bdbe7f993924691c8" exitCode=0 Jul 26 18:23:30.191361 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:30.190988 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-614abb29-xwh8l" event={"ID":"e1b42278-92cd-46ca-ac35-c73d52dbc28e","Type":"ContainerDied","Data":"7688abeab8da0bbef29c75baa32fe7957c774158d743708bdbe7f993924691c8"} Jul 26 18:23:31.326971 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:31.326948 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-614abb29-xwh8l" Jul 26 18:23:31.422496 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:31.422463 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1b42278-92cd-46ca-ac35-c73d52dbc28e-maas-api-service-ca\") pod \"e1b42278-92cd-46ca-ac35-c73d52dbc28e\" (UID: \"e1b42278-92cd-46ca-ac35-c73d52dbc28e\") " Jul 26 18:23:31.422867 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:31.422845 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1b42278-92cd-46ca-ac35-c73d52dbc28e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e1b42278-92cd-46ca-ac35-c73d52dbc28e" (UID: "e1b42278-92cd-46ca-ac35-c73d52dbc28e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:23:31.523349 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:31.523308 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1b42278-92cd-46ca-ac35-c73d52dbc28e-maas-api-service-ca\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:23:32.201840 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:32.201804 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-614abb29-xwh8l" event={"ID":"e1b42278-92cd-46ca-ac35-c73d52dbc28e","Type":"ContainerDied","Data":"21151811e558b77ec8977a57ab36becbf24708f1f220687239110c3462ac0319"} Jul 26 18:23:32.201840 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:32.201834 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-614abb29-xwh8l" Jul 26 18:23:32.202046 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:32.201855 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21151811e558b77ec8977a57ab36becbf24708f1f220687239110c3462ac0319" Jul 26 18:23:43.460340 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:43.460277 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:23:43.576864 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:43.576833 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:23:43.660385 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:43.660356 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:23:48.147581 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:48.147547 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-614abb29-xwh8l"] Jul 26 18:23:48.149983 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:48.149960 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-614abb29-xwh8l"] Jul 26 18:23:48.151117 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:48.151095 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:23:48.439479 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:48.439382 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1b42278-92cd-46ca-ac35-c73d52dbc28e" path="/var/lib/kubelet/pods/e1b42278-92cd-46ca-ac35-c73d52dbc28e/volumes" Jul 26 18:23:51.299309 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:51.299276 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:23:51.631276 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:51.631190 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:23:51.686822 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:51.686785 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:23:56.380251 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:23:56.380207 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:24:32.583567 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:24:32.583537 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:24:33.448856 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:24:33.448818 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:24:38.882517 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:24:38.882483 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:24:44.484782 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:24:44.484714 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:24:44.585596 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:24:44.585564 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:24:44.791318 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:24:44.791235 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:24:45.018371 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:24:45.018326 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:24:46.044685 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:24:46.044640 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:24:46.143918 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:24:46.143887 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:24:46.243789 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:24:46.243758 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:24:47.046129 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:24:47.046097 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:24:47.173217 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:24:47.173182 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:24:47.300569 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:24:47.300481 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:24:48.642710 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:24:48.642656 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:24:54.579347 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:24:54.579308 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:24:54.679198 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:24:54.679158 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:24:55.255979 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:24:55.255941 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:01.100591 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:01.100511 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:01.208155 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:01.208121 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:01.296530 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:01.296496 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:11.796764 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:11.796728 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:12.151551 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:12.151459 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:33.269135 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:33.269101 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:39.407207 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:39.407170 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:39.508735 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:39.508700 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:39.751221 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:39.751189 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:40.065123 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:40.065048 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:40.272478 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:40.272448 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:43.311567 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:43.311527 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:43.406390 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:43.406352 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:43.506020 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:43.505985 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:43.829893 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:43.829865 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:44.093175 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:44.093092 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:44.182402 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:44.182372 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:44.264064 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:44.264032 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c"] Jul 26 18:25:44.264354 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:44.264342 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1b42278-92cd-46ca-ac35-c73d52dbc28e" containerName="revoke-keys" Jul 26 18:25:44.264408 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:44.264355 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1b42278-92cd-46ca-ac35-c73d52dbc28e" containerName="revoke-keys" Jul 26 18:25:44.264445 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:44.264418 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1b42278-92cd-46ca-ac35-c73d52dbc28e" containerName="revoke-keys" Jul 26 18:25:44.267187 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:44.267171 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c" Jul 26 18:25:44.269659 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:44.269638 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-bb8f83\"" Jul 26 18:25:44.269797 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:44.269767 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-bb8f83-dockercfg-f4tgs\"" Jul 26 18:25:44.275767 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:44.275739 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c"] Jul 26 18:25:44.287219 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:44.287189 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szpxg\" (UniqueName: \"kubernetes.io/projected/f87bccfa-b4ce-4912-a191-0fcbba4645e4-kube-api-access-szpxg\") pod \"payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c\" (UID: \"f87bccfa-b4ce-4912-a191-0fcbba4645e4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c" Jul 26 18:25:44.287375 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:44.287232 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f87bccfa-b4ce-4912-a191-0fcbba4645e4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c\" (UID: \"f87bccfa-b4ce-4912-a191-0fcbba4645e4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c" Jul 26 18:25:44.387952 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:44.387859 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f87bccfa-b4ce-4912-a191-0fcbba4645e4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c\" (UID: \"f87bccfa-b4ce-4912-a191-0fcbba4645e4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c" Jul 26 18:25:44.388381 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:44.387984 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szpxg\" (UniqueName: \"kubernetes.io/projected/f87bccfa-b4ce-4912-a191-0fcbba4645e4-kube-api-access-szpxg\") pod \"payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c\" (UID: \"f87bccfa-b4ce-4912-a191-0fcbba4645e4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c" Jul 26 18:25:44.388665 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:44.388642 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f87bccfa-b4ce-4912-a191-0fcbba4645e4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c\" (UID: \"f87bccfa-b4ce-4912-a191-0fcbba4645e4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c" Jul 26 18:25:44.396165 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:44.396143 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szpxg\" (UniqueName: \"kubernetes.io/projected/f87bccfa-b4ce-4912-a191-0fcbba4645e4-kube-api-access-szpxg\") pod \"payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c\" (UID: \"f87bccfa-b4ce-4912-a191-0fcbba4645e4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c" Jul 26 18:25:44.578387 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:44.578329 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c" Jul 26 18:25:44.716771 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:44.716727 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c"] Jul 26 18:25:44.718948 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:25:44.718923 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf87bccfa_b4ce_4912_a191_0fcbba4645e4.slice/crio-09314ec343796e7c77012f7bea10b98a15298d42479ec0e666cf0647ef6ee92c WatchSource:0}: Error finding container 09314ec343796e7c77012f7bea10b98a15298d42479ec0e666cf0647ef6ee92c: Status 404 returned error can't find the container with id 09314ec343796e7c77012f7bea10b98a15298d42479ec0e666cf0647ef6ee92c Jul 26 18:25:44.800395 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:44.800354 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c" event={"ID":"f87bccfa-b4ce-4912-a191-0fcbba4645e4","Type":"ContainerStarted","Data":"09314ec343796e7c77012f7bea10b98a15298d42479ec0e666cf0647ef6ee92c"} Jul 26 18:25:44.862967 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:44.862933 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:47.798808 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:47.798775 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:47.818471 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:47.818441 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c" event={"ID":"f87bccfa-b4ce-4912-a191-0fcbba4645e4","Type":"ContainerStarted","Data":"e05d80ffdc579e8e1c1adcce0910756742a86bbb67bc081aa605e19442ca3ca4"} Jul 26 18:25:47.818690 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:47.818511 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c" Jul 26 18:25:47.858619 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:47.858562 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c" podStartSLOduration=1.616865271 podStartE2EDuration="3.858546871s" totalImagesPullingTime="2.2416816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:25:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:25:47.856852383 +0000 UTC m=+2201.809508071" watchObservedRunningTime="2026-07-26 18:25:47.858546871 +0000 UTC m=+2201.811202558" Jul 26 18:25:48.130221 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:48.130126 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:48.427066 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:48.426977 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:48.693944 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:48.693863 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:51.277035 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:51.277004 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:51.393655 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:51.393622 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:51.493872 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:51.493815 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:51.596741 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:51.596631 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:51.970153 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:51.970121 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:52.076310 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:52.076279 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:52.114122 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:52.110658 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg"] Jul 26 18:25:52.120725 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:52.120695 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg" Jul 26 18:25:52.124194 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:52.124164 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-40c67c-dockercfg-x7dhg\"" Jul 26 18:25:52.124194 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:52.124165 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-40c67c\"" Jul 26 18:25:52.125961 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:52.125938 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg"] Jul 26 18:25:52.162081 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:52.162036 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b409089d-6b4f-40fc-9cba-5f15ae9242f3-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg\" (UID: \"b409089d-6b4f-40fc-9cba-5f15ae9242f3\") " pod="openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg" Jul 26 18:25:52.162298 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:52.162111 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjgpn\" (UniqueName: \"kubernetes.io/projected/b409089d-6b4f-40fc-9cba-5f15ae9242f3-kube-api-access-jjgpn\") pod \"payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg\" (UID: \"b409089d-6b4f-40fc-9cba-5f15ae9242f3\") " pod="openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg" Jul 26 18:25:52.263241 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:52.263131 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b409089d-6b4f-40fc-9cba-5f15ae9242f3-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg\" (UID: \"b409089d-6b4f-40fc-9cba-5f15ae9242f3\") " pod="openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg" Jul 26 18:25:52.263241 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:52.263189 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjgpn\" (UniqueName: \"kubernetes.io/projected/b409089d-6b4f-40fc-9cba-5f15ae9242f3-kube-api-access-jjgpn\") pod \"payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg\" (UID: \"b409089d-6b4f-40fc-9cba-5f15ae9242f3\") " pod="openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg" Jul 26 18:25:52.264029 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:52.264008 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b409089d-6b4f-40fc-9cba-5f15ae9242f3-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg\" (UID: \"b409089d-6b4f-40fc-9cba-5f15ae9242f3\") " pod="openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg" Jul 26 18:25:52.271697 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:52.271649 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjgpn\" (UniqueName: \"kubernetes.io/projected/b409089d-6b4f-40fc-9cba-5f15ae9242f3-kube-api-access-jjgpn\") pod \"payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg\" (UID: \"b409089d-6b4f-40fc-9cba-5f15ae9242f3\") " pod="openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg" Jul 26 18:25:52.431831 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:52.431792 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg" Jul 26 18:25:52.571542 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:52.571508 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg"] Jul 26 18:25:52.574927 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:25:52.574889 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb409089d_6b4f_40fc_9cba_5f15ae9242f3.slice/crio-bf85dfbd992fd452faf764faf52e57f0228fdc9ec9d4c2e751f7b7f4d45eb65a WatchSource:0}: Error finding container bf85dfbd992fd452faf764faf52e57f0228fdc9ec9d4c2e751f7b7f4d45eb65a: Status 404 returned error can't find the container with id bf85dfbd992fd452faf764faf52e57f0228fdc9ec9d4c2e751f7b7f4d45eb65a Jul 26 18:25:52.844930 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:52.844831 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg" event={"ID":"b409089d-6b4f-40fc-9cba-5f15ae9242f3","Type":"ContainerStarted","Data":"a1d9fb791f57ca46d28ae57015fdd56430e5cc703da23a1feebfd9cb565bddfb"} Jul 26 18:25:52.844930 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:52.844883 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg" event={"ID":"b409089d-6b4f-40fc-9cba-5f15ae9242f3","Type":"ContainerStarted","Data":"bf85dfbd992fd452faf764faf52e57f0228fdc9ec9d4c2e751f7b7f4d45eb65a"} Jul 26 18:25:52.845132 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:52.844937 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg" Jul 26 18:25:52.861460 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:52.861403 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg" podStartSLOduration=0.861386088 podStartE2EDuration="861.386088ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:25:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:25:52.859657037 +0000 UTC m=+2206.812312752" watchObservedRunningTime="2026-07-26 18:25:52.861386088 +0000 UTC m=+2206.814041775" Jul 26 18:25:55.761127 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:55.761097 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:25:58.824830 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:25:58.824799 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c" Jul 26 18:26:03.851644 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:26:03.851613 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg" Jul 26 18:26:16.418039 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:26:16.417998 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:26:44.197219 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:26:44.197124 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:27:25.517654 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:25.517619 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:27:26.541068 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:26.541036 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bb8f83-chr8h"] Jul 26 18:27:26.544502 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:26.544483 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bb8f83-chr8h" Jul 26 18:27:26.546789 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:26.546768 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-swzk5\"" Jul 26 18:27:26.551144 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:26.551119 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bb8f83-chr8h"] Jul 26 18:27:26.616630 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:26.616586 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93f21be9-2229-4f32-8f99-1bbefb49dd3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-bb8f83-chr8h\" (UID: \"93f21be9-2229-4f32-8f99-1bbefb49dd3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bb8f83-chr8h" Jul 26 18:27:26.717903 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:26.717871 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93f21be9-2229-4f32-8f99-1bbefb49dd3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-bb8f83-chr8h\" (UID: \"93f21be9-2229-4f32-8f99-1bbefb49dd3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bb8f83-chr8h" Jul 26 18:27:26.718438 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:26.718419 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93f21be9-2229-4f32-8f99-1bbefb49dd3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-bb8f83-chr8h\" (UID: \"93f21be9-2229-4f32-8f99-1bbefb49dd3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bb8f83-chr8h" Jul 26 18:27:26.733082 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:26.733057 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:27:26.857574 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:26.857490 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bb8f83-chr8h" Jul 26 18:27:26.992016 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:26.991992 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bb8f83-chr8h"] Jul 26 18:27:26.994544 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:27:26.994518 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93f21be9_2229_4f32_8f99_1bbefb49dd3b.slice/crio-ddf26805f2a7315488030953f43ab3b65af0d70e545d373045a3c9778f410bca WatchSource:0}: Error finding container ddf26805f2a7315488030953f43ab3b65af0d70e545d373045a3c9778f410bca: Status 404 returned error can't find the container with id ddf26805f2a7315488030953f43ab3b65af0d70e545d373045a3c9778f410bca Jul 26 18:27:27.273530 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:27.273497 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bb8f83-chr8h" event={"ID":"93f21be9-2229-4f32-8f99-1bbefb49dd3b","Type":"ContainerStarted","Data":"ddf26805f2a7315488030953f43ab3b65af0d70e545d373045a3c9778f410bca"} Jul 26 18:27:28.066373 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:28.066289 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:27:28.279689 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:28.279634 2559 generic.go:356] "Generic (PLEG): container finished" podID="93f21be9-2229-4f32-8f99-1bbefb49dd3b" containerID="2f49ab434471a10c317ff229653848be39a3e4c15d8e97fc0ea6684b46bb98eb" exitCode=0 Jul 26 18:27:28.279853 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:28.279710 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bb8f83-chr8h" event={"ID":"93f21be9-2229-4f32-8f99-1bbefb49dd3b","Type":"ContainerDied","Data":"2f49ab434471a10c317ff229653848be39a3e4c15d8e97fc0ea6684b46bb98eb"} Jul 26 18:27:29.417322 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:29.417295 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bb8f83-chr8h" Jul 26 18:27:29.539740 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:29.539621 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93f21be9-2229-4f32-8f99-1bbefb49dd3b-maas-api-service-ca\") pod \"93f21be9-2229-4f32-8f99-1bbefb49dd3b\" (UID: \"93f21be9-2229-4f32-8f99-1bbefb49dd3b\") " Jul 26 18:27:29.540100 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:29.540074 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93f21be9-2229-4f32-8f99-1bbefb49dd3b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "93f21be9-2229-4f32-8f99-1bbefb49dd3b" (UID: "93f21be9-2229-4f32-8f99-1bbefb49dd3b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:27:29.640459 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:29.640428 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/93f21be9-2229-4f32-8f99-1bbefb49dd3b-maas-api-service-ca\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:27:30.289836 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:30.289807 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bb8f83-chr8h" Jul 26 18:27:30.290020 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:30.289806 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bb8f83-chr8h" event={"ID":"93f21be9-2229-4f32-8f99-1bbefb49dd3b","Type":"ContainerDied","Data":"ddf26805f2a7315488030953f43ab3b65af0d70e545d373045a3c9778f410bca"} Jul 26 18:27:30.290020 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:30.289926 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ddf26805f2a7315488030953f43ab3b65af0d70e545d373045a3c9778f410bca" Jul 26 18:27:36.968330 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:36.968296 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:27:37.009423 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:37.009387 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c"] Jul 26 18:27:37.009630 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:37.009599 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c" podUID="f87bccfa-b4ce-4912-a191-0fcbba4645e4" containerName="payload-pre-processing" containerID="cri-o://e05d80ffdc579e8e1c1adcce0910756742a86bbb67bc081aa605e19442ca3ca4" gracePeriod=30 Jul 26 18:27:37.070103 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:37.070068 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:27:37.170658 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:37.170614 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:27:37.268129 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:37.268107 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c" Jul 26 18:27:37.325942 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:37.325894 2559 generic.go:356] "Generic (PLEG): container finished" podID="f87bccfa-b4ce-4912-a191-0fcbba4645e4" containerID="e05d80ffdc579e8e1c1adcce0910756742a86bbb67bc081aa605e19442ca3ca4" exitCode=0 Jul 26 18:27:37.325942 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:37.325930 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c" Jul 26 18:27:37.326135 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:37.325952 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c" event={"ID":"f87bccfa-b4ce-4912-a191-0fcbba4645e4","Type":"ContainerDied","Data":"e05d80ffdc579e8e1c1adcce0910756742a86bbb67bc081aa605e19442ca3ca4"} Jul 26 18:27:37.326135 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:37.326015 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c" event={"ID":"f87bccfa-b4ce-4912-a191-0fcbba4645e4","Type":"ContainerDied","Data":"09314ec343796e7c77012f7bea10b98a15298d42479ec0e666cf0647ef6ee92c"} Jul 26 18:27:37.326135 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:37.326047 2559 scope.go:117] "RemoveContainer" containerID="e05d80ffdc579e8e1c1adcce0910756742a86bbb67bc081aa605e19442ca3ca4" Jul 26 18:27:37.334125 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:37.334099 2559 scope.go:117] "RemoveContainer" containerID="e05d80ffdc579e8e1c1adcce0910756742a86bbb67bc081aa605e19442ca3ca4" Jul 26 18:27:37.334379 ip-10-0-128-209 kubenswrapper[2559]: E0726 18:27:37.334359 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e05d80ffdc579e8e1c1adcce0910756742a86bbb67bc081aa605e19442ca3ca4\": container with ID starting with e05d80ffdc579e8e1c1adcce0910756742a86bbb67bc081aa605e19442ca3ca4 not found: ID does not exist" containerID="e05d80ffdc579e8e1c1adcce0910756742a86bbb67bc081aa605e19442ca3ca4" Jul 26 18:27:37.334440 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:37.334387 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e05d80ffdc579e8e1c1adcce0910756742a86bbb67bc081aa605e19442ca3ca4"} err="failed to get container status \"e05d80ffdc579e8e1c1adcce0910756742a86bbb67bc081aa605e19442ca3ca4\": rpc error: code = NotFound desc = could not find container \"e05d80ffdc579e8e1c1adcce0910756742a86bbb67bc081aa605e19442ca3ca4\": container with ID starting with e05d80ffdc579e8e1c1adcce0910756742a86bbb67bc081aa605e19442ca3ca4 not found: ID does not exist" Jul 26 18:27:37.412097 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:37.412062 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f87bccfa-b4ce-4912-a191-0fcbba4645e4-plugins-config-volume\") pod \"f87bccfa-b4ce-4912-a191-0fcbba4645e4\" (UID: \"f87bccfa-b4ce-4912-a191-0fcbba4645e4\") " Jul 26 18:27:37.412294 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:37.412125 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szpxg\" (UniqueName: \"kubernetes.io/projected/f87bccfa-b4ce-4912-a191-0fcbba4645e4-kube-api-access-szpxg\") pod \"f87bccfa-b4ce-4912-a191-0fcbba4645e4\" (UID: \"f87bccfa-b4ce-4912-a191-0fcbba4645e4\") " Jul 26 18:27:37.412450 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:37.412426 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f87bccfa-b4ce-4912-a191-0fcbba4645e4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f87bccfa-b4ce-4912-a191-0fcbba4645e4" (UID: "f87bccfa-b4ce-4912-a191-0fcbba4645e4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:27:37.414294 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:37.414268 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f87bccfa-b4ce-4912-a191-0fcbba4645e4-kube-api-access-szpxg" (OuterVolumeSpecName: "kube-api-access-szpxg") pod "f87bccfa-b4ce-4912-a191-0fcbba4645e4" (UID: "f87bccfa-b4ce-4912-a191-0fcbba4645e4"). InnerVolumeSpecName "kube-api-access-szpxg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:27:37.513118 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:37.513013 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f87bccfa-b4ce-4912-a191-0fcbba4645e4-plugins-config-volume\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:27:37.513118 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:37.513044 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szpxg\" (UniqueName: \"kubernetes.io/projected/f87bccfa-b4ce-4912-a191-0fcbba4645e4-kube-api-access-szpxg\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:27:37.647310 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:37.647272 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c"] Jul 26 18:27:37.653498 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:37.653473 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bb8f83-5c6dd89f6-lng6c"] Jul 26 18:27:38.438752 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:38.438720 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f87bccfa-b4ce-4912-a191-0fcbba4645e4" path="/var/lib/kubelet/pods/f87bccfa-b4ce-4912-a191-0fcbba4645e4/volumes" Jul 26 18:27:41.622006 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:41.621954 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:27:41.638331 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:41.638299 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bb8f83-chr8h"] Jul 26 18:27:41.642126 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:41.642098 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-bb8f83-chr8h"] Jul 26 18:27:41.999487 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:41.999455 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:27:42.438521 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:42.438440 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93f21be9-2229-4f32-8f99-1bbefb49dd3b" path="/var/lib/kubelet/pods/93f21be9-2229-4f32-8f99-1bbefb49dd3b/volumes" Jul 26 18:27:47.873142 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:47.873107 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:27:47.973529 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:47.973490 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:27:48.122111 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:48.122054 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:27:48.340836 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:48.340795 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:27:58.892187 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:58.892117 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:27:58.916428 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:58.916393 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg"] Jul 26 18:27:58.916663 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:58.916636 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg" podUID="b409089d-6b4f-40fc-9cba-5f15ae9242f3" containerName="payload-processing" containerID="cri-o://a1d9fb791f57ca46d28ae57015fdd56430e5cc703da23a1feebfd9cb565bddfb" gracePeriod=30 Jul 26 18:27:58.992416 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:58.992387 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:27:59.093966 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:59.093929 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:27:59.164967 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:59.161992 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg" Jul 26 18:27:59.212695 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:59.212633 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjgpn\" (UniqueName: \"kubernetes.io/projected/b409089d-6b4f-40fc-9cba-5f15ae9242f3-kube-api-access-jjgpn\") pod \"b409089d-6b4f-40fc-9cba-5f15ae9242f3\" (UID: \"b409089d-6b4f-40fc-9cba-5f15ae9242f3\") " Jul 26 18:27:59.212911 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:59.212747 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b409089d-6b4f-40fc-9cba-5f15ae9242f3-plugins-config-volume\") pod \"b409089d-6b4f-40fc-9cba-5f15ae9242f3\" (UID: \"b409089d-6b4f-40fc-9cba-5f15ae9242f3\") " Jul 26 18:27:59.213207 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:59.213184 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b409089d-6b4f-40fc-9cba-5f15ae9242f3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b409089d-6b4f-40fc-9cba-5f15ae9242f3" (UID: "b409089d-6b4f-40fc-9cba-5f15ae9242f3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:27:59.215190 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:59.215167 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b409089d-6b4f-40fc-9cba-5f15ae9242f3-kube-api-access-jjgpn" (OuterVolumeSpecName: "kube-api-access-jjgpn") pod "b409089d-6b4f-40fc-9cba-5f15ae9242f3" (UID: "b409089d-6b4f-40fc-9cba-5f15ae9242f3"). InnerVolumeSpecName "kube-api-access-jjgpn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:27:59.313931 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:59.313894 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjgpn\" (UniqueName: \"kubernetes.io/projected/b409089d-6b4f-40fc-9cba-5f15ae9242f3-kube-api-access-jjgpn\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:27:59.313931 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:59.313926 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b409089d-6b4f-40fc-9cba-5f15ae9242f3-plugins-config-volume\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:27:59.425631 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:59.425539 2559 generic.go:356] "Generic (PLEG): container finished" podID="b409089d-6b4f-40fc-9cba-5f15ae9242f3" containerID="a1d9fb791f57ca46d28ae57015fdd56430e5cc703da23a1feebfd9cb565bddfb" exitCode=0 Jul 26 18:27:59.425631 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:59.425584 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg" Jul 26 18:27:59.425851 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:59.425627 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg" event={"ID":"b409089d-6b4f-40fc-9cba-5f15ae9242f3","Type":"ContainerDied","Data":"a1d9fb791f57ca46d28ae57015fdd56430e5cc703da23a1feebfd9cb565bddfb"} Jul 26 18:27:59.425851 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:59.425722 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg" event={"ID":"b409089d-6b4f-40fc-9cba-5f15ae9242f3","Type":"ContainerDied","Data":"bf85dfbd992fd452faf764faf52e57f0228fdc9ec9d4c2e751f7b7f4d45eb65a"} Jul 26 18:27:59.425851 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:59.425747 2559 scope.go:117] "RemoveContainer" containerID="a1d9fb791f57ca46d28ae57015fdd56430e5cc703da23a1feebfd9cb565bddfb" Jul 26 18:27:59.434072 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:59.434054 2559 scope.go:117] "RemoveContainer" containerID="a1d9fb791f57ca46d28ae57015fdd56430e5cc703da23a1feebfd9cb565bddfb" Jul 26 18:27:59.434346 ip-10-0-128-209 kubenswrapper[2559]: E0726 18:27:59.434327 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1d9fb791f57ca46d28ae57015fdd56430e5cc703da23a1feebfd9cb565bddfb\": container with ID starting with a1d9fb791f57ca46d28ae57015fdd56430e5cc703da23a1feebfd9cb565bddfb not found: ID does not exist" containerID="a1d9fb791f57ca46d28ae57015fdd56430e5cc703da23a1feebfd9cb565bddfb" Jul 26 18:27:59.434397 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:59.434352 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1d9fb791f57ca46d28ae57015fdd56430e5cc703da23a1feebfd9cb565bddfb"} err="failed to get container status \"a1d9fb791f57ca46d28ae57015fdd56430e5cc703da23a1feebfd9cb565bddfb\": rpc error: code = NotFound desc = could not find container \"a1d9fb791f57ca46d28ae57015fdd56430e5cc703da23a1feebfd9cb565bddfb\": container with ID starting with a1d9fb791f57ca46d28ae57015fdd56430e5cc703da23a1feebfd9cb565bddfb not found: ID does not exist" Jul 26 18:27:59.446300 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:59.446273 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg"] Jul 26 18:27:59.451684 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:27:59.451654 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-40c67c-76b5478b6b-qxvfg"] Jul 26 18:28:00.438410 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:00.438378 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b409089d-6b4f-40fc-9cba-5f15ae9242f3" path="/var/lib/kubelet/pods/b409089d-6b4f-40fc-9cba-5f15ae9242f3/volumes" Jul 26 18:28:03.528516 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:03.528483 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:03.909401 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:03.909316 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:09.810775 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:09.810742 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:09.911272 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:09.911236 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:10.082629 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:10.082544 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:10.287233 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:10.287202 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:10.618029 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:10.617996 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:10.915496 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:10.915403 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:11.513874 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:11.513841 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:14.071543 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.071512 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:14.166196 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.166158 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:14.239583 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.239555 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:14.339209 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.339114 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:14.854283 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.854244 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:14.952542 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.952505 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:14.976185 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.976155 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm"] Jul 26 18:28:14.976502 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.976487 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93f21be9-2229-4f32-8f99-1bbefb49dd3b" containerName="revoke-keys" Jul 26 18:28:14.976547 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.976505 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="93f21be9-2229-4f32-8f99-1bbefb49dd3b" containerName="revoke-keys" Jul 26 18:28:14.976547 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.976520 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f87bccfa-b4ce-4912-a191-0fcbba4645e4" containerName="payload-pre-processing" Jul 26 18:28:14.976547 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.976525 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f87bccfa-b4ce-4912-a191-0fcbba4645e4" containerName="payload-pre-processing" Jul 26 18:28:14.976547 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.976534 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b409089d-6b4f-40fc-9cba-5f15ae9242f3" containerName="payload-processing" Jul 26 18:28:14.976547 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.976539 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b409089d-6b4f-40fc-9cba-5f15ae9242f3" containerName="payload-processing" Jul 26 18:28:14.976730 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.976589 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b409089d-6b4f-40fc-9cba-5f15ae9242f3" containerName="payload-processing" Jul 26 18:28:14.976730 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.976596 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="93f21be9-2229-4f32-8f99-1bbefb49dd3b" containerName="revoke-keys" Jul 26 18:28:14.976730 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.976603 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f87bccfa-b4ce-4912-a191-0fcbba4645e4" containerName="payload-pre-processing" Jul 26 18:28:14.979450 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.979430 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm" Jul 26 18:28:14.982506 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.982490 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-b6e870\"" Jul 26 18:28:14.991036 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.991016 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6"] Jul 26 18:28:14.994164 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.994140 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6" Jul 26 18:28:14.997267 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.997241 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm"] Jul 26 18:28:14.997966 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:14.997946 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-b6e870-dockercfg-lhnr5\"" Jul 26 18:28:15.013400 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:15.012523 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6"] Jul 26 18:28:15.053506 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:15.053471 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/308b1218-6b2e-434d-bb96-c506386d75b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm\" (UID: \"308b1218-6b2e-434d-bb96-c506386d75b9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm" Jul 26 18:28:15.053506 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:15.053506 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db9jr\" (UniqueName: \"kubernetes.io/projected/934a903f-5ccf-417c-bb83-1cddceda94dd-kube-api-access-db9jr\") pod \"payload-processing-e2e-api-a-b6e870-797d645956-txdf6\" (UID: \"934a903f-5ccf-417c-bb83-1cddceda94dd\") " pod="openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6" Jul 26 18:28:15.053742 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:15.053545 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/934a903f-5ccf-417c-bb83-1cddceda94dd-plugins-config-volume\") pod \"payload-processing-e2e-api-a-b6e870-797d645956-txdf6\" (UID: \"934a903f-5ccf-417c-bb83-1cddceda94dd\") " pod="openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6" Jul 26 18:28:15.053742 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:15.053589 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgfx8\" (UniqueName: \"kubernetes.io/projected/308b1218-6b2e-434d-bb96-c506386d75b9-kube-api-access-bgfx8\") pod \"payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm\" (UID: \"308b1218-6b2e-434d-bb96-c506386d75b9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm" Jul 26 18:28:15.154276 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:15.154189 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/934a903f-5ccf-417c-bb83-1cddceda94dd-plugins-config-volume\") pod \"payload-processing-e2e-api-a-b6e870-797d645956-txdf6\" (UID: \"934a903f-5ccf-417c-bb83-1cddceda94dd\") " pod="openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6" Jul 26 18:28:15.154276 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:15.154244 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgfx8\" (UniqueName: \"kubernetes.io/projected/308b1218-6b2e-434d-bb96-c506386d75b9-kube-api-access-bgfx8\") pod \"payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm\" (UID: \"308b1218-6b2e-434d-bb96-c506386d75b9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm" Jul 26 18:28:15.154653 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:15.154301 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/308b1218-6b2e-434d-bb96-c506386d75b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm\" (UID: \"308b1218-6b2e-434d-bb96-c506386d75b9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm" Jul 26 18:28:15.154653 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:15.154317 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db9jr\" (UniqueName: \"kubernetes.io/projected/934a903f-5ccf-417c-bb83-1cddceda94dd-kube-api-access-db9jr\") pod \"payload-processing-e2e-api-a-b6e870-797d645956-txdf6\" (UID: \"934a903f-5ccf-417c-bb83-1cddceda94dd\") " pod="openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6" Jul 26 18:28:15.154995 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:15.154974 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/934a903f-5ccf-417c-bb83-1cddceda94dd-plugins-config-volume\") pod \"payload-processing-e2e-api-a-b6e870-797d645956-txdf6\" (UID: \"934a903f-5ccf-417c-bb83-1cddceda94dd\") " pod="openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6" Jul 26 18:28:15.155051 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:15.155035 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/308b1218-6b2e-434d-bb96-c506386d75b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm\" (UID: \"308b1218-6b2e-434d-bb96-c506386d75b9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm" Jul 26 18:28:15.163385 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:15.163362 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgfx8\" (UniqueName: \"kubernetes.io/projected/308b1218-6b2e-434d-bb96-c506386d75b9-kube-api-access-bgfx8\") pod \"payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm\" (UID: \"308b1218-6b2e-434d-bb96-c506386d75b9\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm" Jul 26 18:28:15.163547 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:15.163527 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-db9jr\" (UniqueName: \"kubernetes.io/projected/934a903f-5ccf-417c-bb83-1cddceda94dd-kube-api-access-db9jr\") pod \"payload-processing-e2e-api-a-b6e870-797d645956-txdf6\" (UID: \"934a903f-5ccf-417c-bb83-1cddceda94dd\") " pod="openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6" Jul 26 18:28:15.289193 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:15.289161 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm" Jul 26 18:28:15.305093 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:15.305054 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6" Jul 26 18:28:15.442124 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:15.441942 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm"] Jul 26 18:28:15.473944 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:15.473725 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6"] Jul 26 18:28:15.476168 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:28:15.476136 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod934a903f_5ccf_417c_bb83_1cddceda94dd.slice/crio-903b28144a0176054d66635e4ae51dc04535220d2d0110971c8711c3cd456544 WatchSource:0}: Error finding container 903b28144a0176054d66635e4ae51dc04535220d2d0110971c8711c3cd456544: Status 404 returned error can't find the container with id 903b28144a0176054d66635e4ae51dc04535220d2d0110971c8711c3cd456544 Jul 26 18:28:15.498700 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:15.498654 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6" event={"ID":"934a903f-5ccf-417c-bb83-1cddceda94dd","Type":"ContainerStarted","Data":"903b28144a0176054d66635e4ae51dc04535220d2d0110971c8711c3cd456544"} Jul 26 18:28:15.499734 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:15.499705 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm" event={"ID":"308b1218-6b2e-434d-bb96-c506386d75b9","Type":"ContainerStarted","Data":"06136e661777fc81b1194d13bcf0b5ffe16f3ca57289405c79875713a9dd0b09"} Jul 26 18:28:16.505539 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:16.505502 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6" event={"ID":"934a903f-5ccf-417c-bb83-1cddceda94dd","Type":"ContainerStarted","Data":"443041d108853a7ea349009c67ec1c418f2ddb5bc3c6b5dd5fe205075c1c1dc5"} Jul 26 18:28:16.506056 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:16.505625 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6" Jul 26 18:28:16.507378 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:16.507347 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm" event={"ID":"308b1218-6b2e-434d-bb96-c506386d75b9","Type":"ContainerStarted","Data":"82fe44fa961f8b439d037a6055c1eb5621cc24a06226bac9af6004e2d037ba21"} Jul 26 18:28:16.507534 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:16.507432 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm" Jul 26 18:28:16.537647 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:16.537573 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm" podStartSLOduration=2.537556847 podStartE2EDuration="2.537556847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:28:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:28:16.535947077 +0000 UTC m=+2350.488602765" watchObservedRunningTime="2026-07-26 18:28:16.537556847 +0000 UTC m=+2350.490212534" Jul 26 18:28:16.538013 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:16.537992 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6" podStartSLOduration=2.537983899 podStartE2EDuration="2.537983899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:28:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:28:16.521372993 +0000 UTC m=+2350.474028683" watchObservedRunningTime="2026-07-26 18:28:16.537983899 +0000 UTC m=+2350.490639586" Jul 26 18:28:18.554905 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:18.554868 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:19.000011 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:18.999978 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:19.306985 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:19.306893 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:19.506857 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:19.506750 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:22.200686 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:22.200640 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:22.294991 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:22.294953 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:22.357402 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:22.357366 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:22.831568 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:22.831536 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:22.929928 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:22.929894 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:22.939788 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:22.939759 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w"] Jul 26 18:28:22.943472 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:22.943448 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w" Jul 26 18:28:22.946112 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:22.946092 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-167468\"" Jul 26 18:28:22.946390 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:22.946373 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-167468-dockercfg-98bnd\"" Jul 26 18:28:22.956512 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:22.956487 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w"] Jul 26 18:28:22.972531 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:22.972496 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz"] Jul 26 18:28:22.976203 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:22.976182 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz" Jul 26 18:28:22.986778 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:22.986751 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz"] Jul 26 18:28:23.028546 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.028505 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/026b023f-15f3-4706-8752-1d9920f054ea-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w\" (UID: \"026b023f-15f3-4706-8752-1d9920f054ea\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w" Jul 26 18:28:23.028546 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.028555 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxs2p\" (UniqueName: \"kubernetes.io/projected/026b023f-15f3-4706-8752-1d9920f054ea-kube-api-access-sxs2p\") pod \"payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w\" (UID: \"026b023f-15f3-4706-8752-1d9920f054ea\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w" Jul 26 18:28:23.130036 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.129944 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/026b023f-15f3-4706-8752-1d9920f054ea-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w\" (UID: \"026b023f-15f3-4706-8752-1d9920f054ea\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w" Jul 26 18:28:23.130036 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.129981 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxs2p\" (UniqueName: \"kubernetes.io/projected/026b023f-15f3-4706-8752-1d9920f054ea-kube-api-access-sxs2p\") pod \"payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w\" (UID: \"026b023f-15f3-4706-8752-1d9920f054ea\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w" Jul 26 18:28:23.130285 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.130041 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa-plugins-config-volume\") pod \"payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz\" (UID: \"5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa\") " pod="openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz" Jul 26 18:28:23.130285 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.130066 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdz69\" (UniqueName: \"kubernetes.io/projected/5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa-kube-api-access-fdz69\") pod \"payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz\" (UID: \"5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa\") " pod="openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz" Jul 26 18:28:23.130781 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.130756 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/026b023f-15f3-4706-8752-1d9920f054ea-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w\" (UID: \"026b023f-15f3-4706-8752-1d9920f054ea\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w" Jul 26 18:28:23.138553 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.138524 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxs2p\" (UniqueName: \"kubernetes.io/projected/026b023f-15f3-4706-8752-1d9920f054ea-kube-api-access-sxs2p\") pod \"payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w\" (UID: \"026b023f-15f3-4706-8752-1d9920f054ea\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w" Jul 26 18:28:23.230783 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.230738 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa-plugins-config-volume\") pod \"payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz\" (UID: \"5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa\") " pod="openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz" Jul 26 18:28:23.230783 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.230788 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdz69\" (UniqueName: \"kubernetes.io/projected/5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa-kube-api-access-fdz69\") pod \"payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz\" (UID: \"5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa\") " pod="openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz" Jul 26 18:28:23.231503 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.231481 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa-plugins-config-volume\") pod \"payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz\" (UID: \"5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa\") " pod="openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz" Jul 26 18:28:23.240048 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.240020 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdz69\" (UniqueName: \"kubernetes.io/projected/5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa-kube-api-access-fdz69\") pod \"payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz\" (UID: \"5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa\") " pod="openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz" Jul 26 18:28:23.255413 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.255377 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w" Jul 26 18:28:23.288471 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.288441 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz" Jul 26 18:28:23.315438 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.315239 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:23.406157 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.406025 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w"] Jul 26 18:28:23.408393 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:28:23.408141 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod026b023f_15f3_4706_8752_1d9920f054ea.slice/crio-f4323ccf1c51a36641dc93199ae70b75f8bf1e33db73a8c14b36c128ff605102 WatchSource:0}: Error finding container f4323ccf1c51a36641dc93199ae70b75f8bf1e33db73a8c14b36c128ff605102: Status 404 returned error can't find the container with id f4323ccf1c51a36641dc93199ae70b75f8bf1e33db73a8c14b36c128ff605102 Jul 26 18:28:23.446003 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.445979 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz"] Jul 26 18:28:23.448658 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:28:23.448627 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5beb19e4_d9cd_4cec_8a2c_7af0f49bc8fa.slice/crio-38ce94d49fd923452178fd0ccfcf092c1af7329d923b3d1b898e30db9f2f818b WatchSource:0}: Error finding container 38ce94d49fd923452178fd0ccfcf092c1af7329d923b3d1b898e30db9f2f818b: Status 404 returned error can't find the container with id 38ce94d49fd923452178fd0ccfcf092c1af7329d923b3d1b898e30db9f2f818b Jul 26 18:28:23.543557 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.543521 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w" event={"ID":"026b023f-15f3-4706-8752-1d9920f054ea","Type":"ContainerStarted","Data":"db3cbad54b85b95f13db5ebb52fe78219cd110ec669ea3bfcf600f00095afa50"} Jul 26 18:28:23.543776 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.543574 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w" event={"ID":"026b023f-15f3-4706-8752-1d9920f054ea","Type":"ContainerStarted","Data":"f4323ccf1c51a36641dc93199ae70b75f8bf1e33db73a8c14b36c128ff605102"} Jul 26 18:28:23.543776 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.543636 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w" Jul 26 18:28:23.545148 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.545124 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz" event={"ID":"5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa","Type":"ContainerStarted","Data":"dfcb346246d28d86d78af52f952cb0220a534db2ba9bc36d33781a3b6fa0b88f"} Jul 26 18:28:23.545238 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.545166 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz" event={"ID":"5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa","Type":"ContainerStarted","Data":"38ce94d49fd923452178fd0ccfcf092c1af7329d923b3d1b898e30db9f2f818b"} Jul 26 18:28:23.545238 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.545227 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz" Jul 26 18:28:23.560528 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.560472 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w" podStartSLOduration=1.56045786 podStartE2EDuration="1.56045786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:28:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:28:23.558543506 +0000 UTC m=+2357.511199223" watchObservedRunningTime="2026-07-26 18:28:23.56045786 +0000 UTC m=+2357.513113548" Jul 26 18:28:23.576418 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:23.576365 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz" podStartSLOduration=1.576348194 podStartE2EDuration="1.576348194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:28:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:28:23.57439139 +0000 UTC m=+2357.527047099" watchObservedRunningTime="2026-07-26 18:28:23.576348194 +0000 UTC m=+2357.529003964" Jul 26 18:28:26.577842 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:26.577810 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:27.515266 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:27.515234 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6" Jul 26 18:28:27.515450 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:27.515322 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm" Jul 26 18:28:34.553011 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:34.552982 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz" Jul 26 18:28:34.553468 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:34.553050 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w" Jul 26 18:28:49.718387 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:49.718351 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:49.749620 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:49.749587 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6"] Jul 26 18:28:49.749841 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:49.749818 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6" podUID="934a903f-5ccf-417c-bb83-1cddceda94dd" containerName="payload-processing" containerID="cri-o://443041d108853a7ea349009c67ec1c418f2ddb5bc3c6b5dd5fe205075c1c1dc5" gracePeriod=30 Jul 26 18:28:49.757208 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:49.757185 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm"] Jul 26 18:28:49.757439 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:49.757411 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm" podUID="308b1218-6b2e-434d-bb96-c506386d75b9" containerName="payload-pre-processing" containerID="cri-o://82fe44fa961f8b439d037a6055c1eb5621cc24a06226bac9af6004e2d037ba21" gracePeriod=30 Jul 26 18:28:49.788431 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:49.788402 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:49.846464 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:49.846428 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:50.029093 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.029038 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6" Jul 26 18:28:50.031963 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.031942 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm" Jul 26 18:28:50.072531 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.072501 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgfx8\" (UniqueName: \"kubernetes.io/projected/308b1218-6b2e-434d-bb96-c506386d75b9-kube-api-access-bgfx8\") pod \"308b1218-6b2e-434d-bb96-c506386d75b9\" (UID: \"308b1218-6b2e-434d-bb96-c506386d75b9\") " Jul 26 18:28:50.072717 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.072589 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-db9jr\" (UniqueName: \"kubernetes.io/projected/934a903f-5ccf-417c-bb83-1cddceda94dd-kube-api-access-db9jr\") pod \"934a903f-5ccf-417c-bb83-1cddceda94dd\" (UID: \"934a903f-5ccf-417c-bb83-1cddceda94dd\") " Jul 26 18:28:50.072717 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.072612 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/308b1218-6b2e-434d-bb96-c506386d75b9-plugins-config-volume\") pod \"308b1218-6b2e-434d-bb96-c506386d75b9\" (UID: \"308b1218-6b2e-434d-bb96-c506386d75b9\") " Jul 26 18:28:50.072717 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.072637 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/934a903f-5ccf-417c-bb83-1cddceda94dd-plugins-config-volume\") pod \"934a903f-5ccf-417c-bb83-1cddceda94dd\" (UID: \"934a903f-5ccf-417c-bb83-1cddceda94dd\") " Jul 26 18:28:50.073147 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.073116 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/308b1218-6b2e-434d-bb96-c506386d75b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "308b1218-6b2e-434d-bb96-c506386d75b9" (UID: "308b1218-6b2e-434d-bb96-c506386d75b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:28:50.073147 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.073137 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/934a903f-5ccf-417c-bb83-1cddceda94dd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "934a903f-5ccf-417c-bb83-1cddceda94dd" (UID: "934a903f-5ccf-417c-bb83-1cddceda94dd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:28:50.074756 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.074723 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/934a903f-5ccf-417c-bb83-1cddceda94dd-kube-api-access-db9jr" (OuterVolumeSpecName: "kube-api-access-db9jr") pod "934a903f-5ccf-417c-bb83-1cddceda94dd" (UID: "934a903f-5ccf-417c-bb83-1cddceda94dd"). InnerVolumeSpecName "kube-api-access-db9jr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:28:50.074864 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.074761 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/308b1218-6b2e-434d-bb96-c506386d75b9-kube-api-access-bgfx8" (OuterVolumeSpecName: "kube-api-access-bgfx8") pod "308b1218-6b2e-434d-bb96-c506386d75b9" (UID: "308b1218-6b2e-434d-bb96-c506386d75b9"). InnerVolumeSpecName "kube-api-access-bgfx8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:28:50.173942 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.173904 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-db9jr\" (UniqueName: \"kubernetes.io/projected/934a903f-5ccf-417c-bb83-1cddceda94dd-kube-api-access-db9jr\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:28:50.173942 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.173937 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/308b1218-6b2e-434d-bb96-c506386d75b9-plugins-config-volume\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:28:50.173942 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.173949 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/934a903f-5ccf-417c-bb83-1cddceda94dd-plugins-config-volume\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:28:50.174171 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.173963 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgfx8\" (UniqueName: \"kubernetes.io/projected/308b1218-6b2e-434d-bb96-c506386d75b9-kube-api-access-bgfx8\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:28:50.678061 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.678029 2559 generic.go:356] "Generic (PLEG): container finished" podID="934a903f-5ccf-417c-bb83-1cddceda94dd" containerID="443041d108853a7ea349009c67ec1c418f2ddb5bc3c6b5dd5fe205075c1c1dc5" exitCode=0 Jul 26 18:28:50.678223 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.678075 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6" Jul 26 18:28:50.678223 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.678112 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6" event={"ID":"934a903f-5ccf-417c-bb83-1cddceda94dd","Type":"ContainerDied","Data":"443041d108853a7ea349009c67ec1c418f2ddb5bc3c6b5dd5fe205075c1c1dc5"} Jul 26 18:28:50.678223 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.678171 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6" event={"ID":"934a903f-5ccf-417c-bb83-1cddceda94dd","Type":"ContainerDied","Data":"903b28144a0176054d66635e4ae51dc04535220d2d0110971c8711c3cd456544"} Jul 26 18:28:50.678223 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.678201 2559 scope.go:117] "RemoveContainer" containerID="443041d108853a7ea349009c67ec1c418f2ddb5bc3c6b5dd5fe205075c1c1dc5" Jul 26 18:28:50.679571 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.679547 2559 generic.go:356] "Generic (PLEG): container finished" podID="308b1218-6b2e-434d-bb96-c506386d75b9" containerID="82fe44fa961f8b439d037a6055c1eb5621cc24a06226bac9af6004e2d037ba21" exitCode=0 Jul 26 18:28:50.679713 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.679609 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm" Jul 26 18:28:50.679787 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.679620 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm" event={"ID":"308b1218-6b2e-434d-bb96-c506386d75b9","Type":"ContainerDied","Data":"82fe44fa961f8b439d037a6055c1eb5621cc24a06226bac9af6004e2d037ba21"} Jul 26 18:28:50.679834 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.679786 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm" event={"ID":"308b1218-6b2e-434d-bb96-c506386d75b9","Type":"ContainerDied","Data":"06136e661777fc81b1194d13bcf0b5ffe16f3ca57289405c79875713a9dd0b09"} Jul 26 18:28:50.687356 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.687340 2559 scope.go:117] "RemoveContainer" containerID="443041d108853a7ea349009c67ec1c418f2ddb5bc3c6b5dd5fe205075c1c1dc5" Jul 26 18:28:50.687613 ip-10-0-128-209 kubenswrapper[2559]: E0726 18:28:50.687597 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"443041d108853a7ea349009c67ec1c418f2ddb5bc3c6b5dd5fe205075c1c1dc5\": container with ID starting with 443041d108853a7ea349009c67ec1c418f2ddb5bc3c6b5dd5fe205075c1c1dc5 not found: ID does not exist" containerID="443041d108853a7ea349009c67ec1c418f2ddb5bc3c6b5dd5fe205075c1c1dc5" Jul 26 18:28:50.687662 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.687621 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"443041d108853a7ea349009c67ec1c418f2ddb5bc3c6b5dd5fe205075c1c1dc5"} err="failed to get container status \"443041d108853a7ea349009c67ec1c418f2ddb5bc3c6b5dd5fe205075c1c1dc5\": rpc error: code = NotFound desc = could not find container \"443041d108853a7ea349009c67ec1c418f2ddb5bc3c6b5dd5fe205075c1c1dc5\": container with ID starting with 443041d108853a7ea349009c67ec1c418f2ddb5bc3c6b5dd5fe205075c1c1dc5 not found: ID does not exist" Jul 26 18:28:50.687662 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.687640 2559 scope.go:117] "RemoveContainer" containerID="82fe44fa961f8b439d037a6055c1eb5621cc24a06226bac9af6004e2d037ba21" Jul 26 18:28:50.695284 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.695264 2559 scope.go:117] "RemoveContainer" containerID="82fe44fa961f8b439d037a6055c1eb5621cc24a06226bac9af6004e2d037ba21" Jul 26 18:28:50.695534 ip-10-0-128-209 kubenswrapper[2559]: E0726 18:28:50.695517 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82fe44fa961f8b439d037a6055c1eb5621cc24a06226bac9af6004e2d037ba21\": container with ID starting with 82fe44fa961f8b439d037a6055c1eb5621cc24a06226bac9af6004e2d037ba21 not found: ID does not exist" containerID="82fe44fa961f8b439d037a6055c1eb5621cc24a06226bac9af6004e2d037ba21" Jul 26 18:28:50.695607 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.695538 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82fe44fa961f8b439d037a6055c1eb5621cc24a06226bac9af6004e2d037ba21"} err="failed to get container status \"82fe44fa961f8b439d037a6055c1eb5621cc24a06226bac9af6004e2d037ba21\": rpc error: code = NotFound desc = could not find container \"82fe44fa961f8b439d037a6055c1eb5621cc24a06226bac9af6004e2d037ba21\": container with ID starting with 82fe44fa961f8b439d037a6055c1eb5621cc24a06226bac9af6004e2d037ba21 not found: ID does not exist" Jul 26 18:28:50.696255 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.696237 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6"] Jul 26 18:28:50.702016 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.701950 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b6e870-797d645956-txdf6"] Jul 26 18:28:50.709639 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.709614 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm"] Jul 26 18:28:50.712872 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:50.712850 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b6e870-69fbdf9b44-c8fnm"] Jul 26 18:28:52.439419 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:52.439386 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="308b1218-6b2e-434d-bb96-c506386d75b9" path="/var/lib/kubelet/pods/308b1218-6b2e-434d-bb96-c506386d75b9/volumes" Jul 26 18:28:52.439802 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:52.439753 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="934a903f-5ccf-417c-bb83-1cddceda94dd" path="/var/lib/kubelet/pods/934a903f-5ccf-417c-bb83-1cddceda94dd/volumes" Jul 26 18:28:54.582605 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:54.582573 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:28:54.961447 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:28:54.961406 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:00.643624 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:00.643575 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:00.744911 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:00.744875 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:00.917326 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:00.917235 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:01.160560 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:01.160520 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:05.989065 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:05.989032 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:06.622227 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:06.622199 2559 scope.go:117] "RemoveContainer" containerID="2dedfb97a98c627ac2f7d28a78aa22b611df48b957659999abdd4a59b7b696b5" Jul 26 18:29:16.696240 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:16.696202 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:16.702609 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:16.702574 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz"] Jul 26 18:29:16.702878 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:16.702831 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz" podUID="5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa" containerName="payload-processing" containerID="cri-o://dfcb346246d28d86d78af52f952cb0220a534db2ba9bc36d33781a3b6fa0b88f" gracePeriod=30 Jul 26 18:29:16.712863 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:16.712829 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w"] Jul 26 18:29:16.713267 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:16.713240 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w" podUID="026b023f-15f3-4706-8752-1d9920f054ea" containerName="payload-pre-processing" containerID="cri-o://db3cbad54b85b95f13db5ebb52fe78219cd110ec669ea3bfcf600f00095afa50" gracePeriod=30 Jul 26 18:29:16.795588 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:16.795538 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:16.998431 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:16.998406 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w" Jul 26 18:29:17.001830 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.001808 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz" Jul 26 18:29:17.117818 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.117782 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/026b023f-15f3-4706-8752-1d9920f054ea-plugins-config-volume\") pod \"026b023f-15f3-4706-8752-1d9920f054ea\" (UID: \"026b023f-15f3-4706-8752-1d9920f054ea\") " Jul 26 18:29:17.117988 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.117828 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdz69\" (UniqueName: \"kubernetes.io/projected/5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa-kube-api-access-fdz69\") pod \"5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa\" (UID: \"5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa\") " Jul 26 18:29:17.117988 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.117863 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxs2p\" (UniqueName: \"kubernetes.io/projected/026b023f-15f3-4706-8752-1d9920f054ea-kube-api-access-sxs2p\") pod \"026b023f-15f3-4706-8752-1d9920f054ea\" (UID: \"026b023f-15f3-4706-8752-1d9920f054ea\") " Jul 26 18:29:17.117988 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.117947 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa-plugins-config-volume\") pod \"5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa\" (UID: \"5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa\") " Jul 26 18:29:17.118106 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.118037 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/026b023f-15f3-4706-8752-1d9920f054ea-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "026b023f-15f3-4706-8752-1d9920f054ea" (UID: "026b023f-15f3-4706-8752-1d9920f054ea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:29:17.118381 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.118361 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/026b023f-15f3-4706-8752-1d9920f054ea-plugins-config-volume\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:29:17.118468 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.118361 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa" (UID: "5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:29:17.120081 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.120060 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa-kube-api-access-fdz69" (OuterVolumeSpecName: "kube-api-access-fdz69") pod "5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa" (UID: "5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa"). InnerVolumeSpecName "kube-api-access-fdz69". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:29:17.120157 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.120101 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/026b023f-15f3-4706-8752-1d9920f054ea-kube-api-access-sxs2p" (OuterVolumeSpecName: "kube-api-access-sxs2p") pod "026b023f-15f3-4706-8752-1d9920f054ea" (UID: "026b023f-15f3-4706-8752-1d9920f054ea"). InnerVolumeSpecName "kube-api-access-sxs2p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:29:17.219546 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.219509 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdz69\" (UniqueName: \"kubernetes.io/projected/5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa-kube-api-access-fdz69\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:29:17.219546 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.219539 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxs2p\" (UniqueName: \"kubernetes.io/projected/026b023f-15f3-4706-8752-1d9920f054ea-kube-api-access-sxs2p\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:29:17.219546 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.219549 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa-plugins-config-volume\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:29:17.807458 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.807427 2559 generic.go:356] "Generic (PLEG): container finished" podID="026b023f-15f3-4706-8752-1d9920f054ea" containerID="db3cbad54b85b95f13db5ebb52fe78219cd110ec669ea3bfcf600f00095afa50" exitCode=0 Jul 26 18:29:17.807915 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.807487 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w" Jul 26 18:29:17.807915 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.807514 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w" event={"ID":"026b023f-15f3-4706-8752-1d9920f054ea","Type":"ContainerDied","Data":"db3cbad54b85b95f13db5ebb52fe78219cd110ec669ea3bfcf600f00095afa50"} Jul 26 18:29:17.807915 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.807577 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w" event={"ID":"026b023f-15f3-4706-8752-1d9920f054ea","Type":"ContainerDied","Data":"f4323ccf1c51a36641dc93199ae70b75f8bf1e33db73a8c14b36c128ff605102"} Jul 26 18:29:17.807915 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.807601 2559 scope.go:117] "RemoveContainer" containerID="db3cbad54b85b95f13db5ebb52fe78219cd110ec669ea3bfcf600f00095afa50" Jul 26 18:29:17.808983 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.808960 2559 generic.go:356] "Generic (PLEG): container finished" podID="5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa" containerID="dfcb346246d28d86d78af52f952cb0220a534db2ba9bc36d33781a3b6fa0b88f" exitCode=0 Jul 26 18:29:17.809053 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.809003 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz" event={"ID":"5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa","Type":"ContainerDied","Data":"dfcb346246d28d86d78af52f952cb0220a534db2ba9bc36d33781a3b6fa0b88f"} Jul 26 18:29:17.809053 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.809015 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz" Jul 26 18:29:17.809053 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.809037 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz" event={"ID":"5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa","Type":"ContainerDied","Data":"38ce94d49fd923452178fd0ccfcf092c1af7329d923b3d1b898e30db9f2f818b"} Jul 26 18:29:17.818048 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.818024 2559 scope.go:117] "RemoveContainer" containerID="db3cbad54b85b95f13db5ebb52fe78219cd110ec669ea3bfcf600f00095afa50" Jul 26 18:29:17.818359 ip-10-0-128-209 kubenswrapper[2559]: E0726 18:29:17.818338 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db3cbad54b85b95f13db5ebb52fe78219cd110ec669ea3bfcf600f00095afa50\": container with ID starting with db3cbad54b85b95f13db5ebb52fe78219cd110ec669ea3bfcf600f00095afa50 not found: ID does not exist" containerID="db3cbad54b85b95f13db5ebb52fe78219cd110ec669ea3bfcf600f00095afa50" Jul 26 18:29:17.818453 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.818364 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db3cbad54b85b95f13db5ebb52fe78219cd110ec669ea3bfcf600f00095afa50"} err="failed to get container status \"db3cbad54b85b95f13db5ebb52fe78219cd110ec669ea3bfcf600f00095afa50\": rpc error: code = NotFound desc = could not find container \"db3cbad54b85b95f13db5ebb52fe78219cd110ec669ea3bfcf600f00095afa50\": container with ID starting with db3cbad54b85b95f13db5ebb52fe78219cd110ec669ea3bfcf600f00095afa50 not found: ID does not exist" Jul 26 18:29:17.818453 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.818379 2559 scope.go:117] "RemoveContainer" containerID="dfcb346246d28d86d78af52f952cb0220a534db2ba9bc36d33781a3b6fa0b88f" Jul 26 18:29:17.827099 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.827077 2559 scope.go:117] "RemoveContainer" containerID="dfcb346246d28d86d78af52f952cb0220a534db2ba9bc36d33781a3b6fa0b88f" Jul 26 18:29:17.827373 ip-10-0-128-209 kubenswrapper[2559]: E0726 18:29:17.827353 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfcb346246d28d86d78af52f952cb0220a534db2ba9bc36d33781a3b6fa0b88f\": container with ID starting with dfcb346246d28d86d78af52f952cb0220a534db2ba9bc36d33781a3b6fa0b88f not found: ID does not exist" containerID="dfcb346246d28d86d78af52f952cb0220a534db2ba9bc36d33781a3b6fa0b88f" Jul 26 18:29:17.827425 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.827380 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfcb346246d28d86d78af52f952cb0220a534db2ba9bc36d33781a3b6fa0b88f"} err="failed to get container status \"dfcb346246d28d86d78af52f952cb0220a534db2ba9bc36d33781a3b6fa0b88f\": rpc error: code = NotFound desc = could not find container \"dfcb346246d28d86d78af52f952cb0220a534db2ba9bc36d33781a3b6fa0b88f\": container with ID starting with dfcb346246d28d86d78af52f952cb0220a534db2ba9bc36d33781a3b6fa0b88f not found: ID does not exist" Jul 26 18:29:17.835035 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.835012 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w"] Jul 26 18:29:17.837249 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.837227 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-167468-588d648f76-bts6w"] Jul 26 18:29:17.847468 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.847440 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz"] Jul 26 18:29:17.851227 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:17.851207 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-167468-5c49fd8b5d-5rxdz"] Jul 26 18:29:18.438745 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:18.438712 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="026b023f-15f3-4706-8752-1d9920f054ea" path="/var/lib/kubelet/pods/026b023f-15f3-4706-8752-1d9920f054ea/volumes" Jul 26 18:29:18.439159 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:18.439138 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa" path="/var/lib/kubelet/pods/5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa/volumes" Jul 26 18:29:21.340500 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:21.340463 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:21.745475 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:21.745442 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:27.673404 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:27.673361 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:27.733741 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:27.733647 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:27.940920 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:27.940837 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:28.154563 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:28.154480 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:28.490358 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:28.490324 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:28.787314 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:28.787204 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:28.907778 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:28.907703 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:31.563526 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:31.563498 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:31.766067 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:31.766039 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:31.861283 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:31.861208 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:32.167011 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.166925 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:32.308806 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.308770 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:32.402363 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.402328 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:32.406292 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.406267 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv"] Jul 26 18:29:32.406618 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.406605 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="026b023f-15f3-4706-8752-1d9920f054ea" containerName="payload-pre-processing" Jul 26 18:29:32.406661 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.406622 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="026b023f-15f3-4706-8752-1d9920f054ea" containerName="payload-pre-processing" Jul 26 18:29:32.406661 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.406633 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa" containerName="payload-processing" Jul 26 18:29:32.406661 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.406640 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa" containerName="payload-processing" Jul 26 18:29:32.406661 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.406647 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="934a903f-5ccf-417c-bb83-1cddceda94dd" containerName="payload-processing" Jul 26 18:29:32.406661 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.406652 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="934a903f-5ccf-417c-bb83-1cddceda94dd" containerName="payload-processing" Jul 26 18:29:32.406873 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.406666 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="308b1218-6b2e-434d-bb96-c506386d75b9" containerName="payload-pre-processing" Jul 26 18:29:32.406873 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.406691 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="308b1218-6b2e-434d-bb96-c506386d75b9" containerName="payload-pre-processing" Jul 26 18:29:32.406873 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.406769 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5beb19e4-d9cd-4cec-8a2c-7af0f49bc8fa" containerName="payload-processing" Jul 26 18:29:32.406873 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.406780 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="308b1218-6b2e-434d-bb96-c506386d75b9" containerName="payload-pre-processing" Jul 26 18:29:32.406873 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.406786 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="026b023f-15f3-4706-8752-1d9920f054ea" containerName="payload-pre-processing" Jul 26 18:29:32.406873 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.406792 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="934a903f-5ccf-417c-bb83-1cddceda94dd" containerName="payload-processing" Jul 26 18:29:32.411530 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.411511 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv" Jul 26 18:29:32.414086 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.414067 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-57f93c-dockercfg-7sfpg\"" Jul 26 18:29:32.414200 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.414074 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-57f93c\"" Jul 26 18:29:32.421526 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.421501 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv"] Jul 26 18:29:32.444358 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.444324 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt"] Jul 26 18:29:32.444538 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.444446 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a573d182-06bd-4ee1-bce8-17ef92641caf-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv\" (UID: \"a573d182-06bd-4ee1-bce8-17ef92641caf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv" Jul 26 18:29:32.444538 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.444492 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9brl\" (UniqueName: \"kubernetes.io/projected/a573d182-06bd-4ee1-bce8-17ef92641caf-kube-api-access-w9brl\") pod \"payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv\" (UID: \"a573d182-06bd-4ee1-bce8-17ef92641caf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv" Jul 26 18:29:32.448060 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.448040 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt" Jul 26 18:29:32.460663 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.460635 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt"] Jul 26 18:29:32.545136 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.545100 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrxdw\" (UniqueName: \"kubernetes.io/projected/e500bed6-796b-408c-8a18-544c385e561e-kube-api-access-jrxdw\") pod \"payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt\" (UID: \"e500bed6-796b-408c-8a18-544c385e561e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt" Jul 26 18:29:32.545326 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.545159 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a573d182-06bd-4ee1-bce8-17ef92641caf-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv\" (UID: \"a573d182-06bd-4ee1-bce8-17ef92641caf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv" Jul 26 18:29:32.545326 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.545224 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e500bed6-796b-408c-8a18-544c385e561e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt\" (UID: \"e500bed6-796b-408c-8a18-544c385e561e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt" Jul 26 18:29:32.545326 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.545264 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9brl\" (UniqueName: \"kubernetes.io/projected/a573d182-06bd-4ee1-bce8-17ef92641caf-kube-api-access-w9brl\") pod \"payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv\" (UID: \"a573d182-06bd-4ee1-bce8-17ef92641caf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv" Jul 26 18:29:32.545913 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.545890 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a573d182-06bd-4ee1-bce8-17ef92641caf-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv\" (UID: \"a573d182-06bd-4ee1-bce8-17ef92641caf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv" Jul 26 18:29:32.554536 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.554508 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9brl\" (UniqueName: \"kubernetes.io/projected/a573d182-06bd-4ee1-bce8-17ef92641caf-kube-api-access-w9brl\") pod \"payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv\" (UID: \"a573d182-06bd-4ee1-bce8-17ef92641caf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv" Jul 26 18:29:32.646123 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.646086 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrxdw\" (UniqueName: \"kubernetes.io/projected/e500bed6-796b-408c-8a18-544c385e561e-kube-api-access-jrxdw\") pod \"payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt\" (UID: \"e500bed6-796b-408c-8a18-544c385e561e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt" Jul 26 18:29:32.646493 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.646134 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e500bed6-796b-408c-8a18-544c385e561e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt\" (UID: \"e500bed6-796b-408c-8a18-544c385e561e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt" Jul 26 18:29:32.646862 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.646842 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e500bed6-796b-408c-8a18-544c385e561e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt\" (UID: \"e500bed6-796b-408c-8a18-544c385e561e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt" Jul 26 18:29:32.657308 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.657280 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrxdw\" (UniqueName: \"kubernetes.io/projected/e500bed6-796b-408c-8a18-544c385e561e-kube-api-access-jrxdw\") pod \"payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt\" (UID: \"e500bed6-796b-408c-8a18-544c385e561e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt" Jul 26 18:29:32.722548 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.722510 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv" Jul 26 18:29:32.757889 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.757850 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt" Jul 26 18:29:32.899304 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.898993 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv"] Jul 26 18:29:32.924513 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:32.924490 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt"] Jul 26 18:29:32.929785 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:29:32.929755 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode500bed6_796b_408c_8a18_544c385e561e.slice/crio-6860ce1074c384751e318420d7fdc7db05b1de2b5942bac10032d90c472e36b1 WatchSource:0}: Error finding container 6860ce1074c384751e318420d7fdc7db05b1de2b5942bac10032d90c472e36b1: Status 404 returned error can't find the container with id 6860ce1074c384751e318420d7fdc7db05b1de2b5942bac10032d90c472e36b1 Jul 26 18:29:33.886205 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:33.886169 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt" event={"ID":"e500bed6-796b-408c-8a18-544c385e561e","Type":"ContainerStarted","Data":"ecc5b6cfc2ff601ea683e4feadc1a08556f4626ba1c13034748da787f0859130"} Jul 26 18:29:33.886660 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:33.886226 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt" event={"ID":"e500bed6-796b-408c-8a18-544c385e561e","Type":"ContainerStarted","Data":"6860ce1074c384751e318420d7fdc7db05b1de2b5942bac10032d90c472e36b1"} Jul 26 18:29:33.886660 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:33.886472 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt" Jul 26 18:29:33.887734 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:33.887711 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv" event={"ID":"a573d182-06bd-4ee1-bce8-17ef92641caf","Type":"ContainerStarted","Data":"83b216af3b7efd0458163a363c1cbf9f9ab3cc5fa685d769232093aab874c5e0"} Jul 26 18:29:33.887834 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:33.887750 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv" event={"ID":"a573d182-06bd-4ee1-bce8-17ef92641caf","Type":"ContainerStarted","Data":"12d12564a18938c81cb20ca6c0f8feb59c1d7d0c4814f23794db1e9a2412261f"} Jul 26 18:29:33.887834 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:33.887809 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv" Jul 26 18:29:33.908778 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:33.908732 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt" podStartSLOduration=1.90871594 podStartE2EDuration="1.90871594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:29:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:29:33.907775743 +0000 UTC m=+2427.860431432" watchObservedRunningTime="2026-07-26 18:29:33.90871594 +0000 UTC m=+2427.861371627" Jul 26 18:29:33.931491 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:33.931439 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv" podStartSLOduration=1.931421597 podStartE2EDuration="1.931421597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:29:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:29:33.930707545 +0000 UTC m=+2427.883363249" watchObservedRunningTime="2026-07-26 18:29:33.931421597 +0000 UTC m=+2427.884077301" Jul 26 18:29:36.015036 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:36.014999 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:36.350043 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:36.349940 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:36.657387 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:36.657301 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:36.817386 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:36.817355 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:39.658595 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:39.658563 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:39.755983 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:39.755946 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:39.832097 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:39.832055 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:40.134298 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.134265 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:40.454360 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.454275 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:40.550812 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.550779 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:40.568716 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.568687 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26"] Jul 26 18:29:40.572264 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.572247 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26" Jul 26 18:29:40.574455 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.574431 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-6aa894-dockercfg-wdl4j\"" Jul 26 18:29:40.574585 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.574525 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-6aa894\"" Jul 26 18:29:40.581727 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.581693 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26"] Jul 26 18:29:40.589159 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.589133 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn"] Jul 26 18:29:40.592319 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.592301 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn" Jul 26 18:29:40.601630 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.601606 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn"] Jul 26 18:29:40.614117 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.614082 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6594d7c-e383-4b97-bb10-2e5a61bb94d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26\" (UID: \"f6594d7c-e383-4b97-bb10-2e5a61bb94d5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26" Jul 26 18:29:40.614277 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.614171 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trw2z\" (UniqueName: \"kubernetes.io/projected/f6594d7c-e383-4b97-bb10-2e5a61bb94d5-kube-api-access-trw2z\") pod \"payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26\" (UID: \"f6594d7c-e383-4b97-bb10-2e5a61bb94d5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26" Jul 26 18:29:40.715350 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.715310 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6594d7c-e383-4b97-bb10-2e5a61bb94d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26\" (UID: \"f6594d7c-e383-4b97-bb10-2e5a61bb94d5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26" Jul 26 18:29:40.715832 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.715383 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d794n\" (UniqueName: \"kubernetes.io/projected/d589aff0-cdbd-43c1-ae2a-445bb9af45d6-kube-api-access-d794n\") pod \"payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn\" (UID: \"d589aff0-cdbd-43c1-ae2a-445bb9af45d6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn" Jul 26 18:29:40.715832 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.715430 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trw2z\" (UniqueName: \"kubernetes.io/projected/f6594d7c-e383-4b97-bb10-2e5a61bb94d5-kube-api-access-trw2z\") pod \"payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26\" (UID: \"f6594d7c-e383-4b97-bb10-2e5a61bb94d5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26" Jul 26 18:29:40.715832 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.715467 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d589aff0-cdbd-43c1-ae2a-445bb9af45d6-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn\" (UID: \"d589aff0-cdbd-43c1-ae2a-445bb9af45d6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn" Jul 26 18:29:40.716130 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.716109 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6594d7c-e383-4b97-bb10-2e5a61bb94d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26\" (UID: \"f6594d7c-e383-4b97-bb10-2e5a61bb94d5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26" Jul 26 18:29:40.724373 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.724339 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trw2z\" (UniqueName: \"kubernetes.io/projected/f6594d7c-e383-4b97-bb10-2e5a61bb94d5-kube-api-access-trw2z\") pod \"payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26\" (UID: \"f6594d7c-e383-4b97-bb10-2e5a61bb94d5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26" Jul 26 18:29:40.816800 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.816768 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d794n\" (UniqueName: \"kubernetes.io/projected/d589aff0-cdbd-43c1-ae2a-445bb9af45d6-kube-api-access-d794n\") pod \"payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn\" (UID: \"d589aff0-cdbd-43c1-ae2a-445bb9af45d6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn" Jul 26 18:29:40.816978 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.816814 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d589aff0-cdbd-43c1-ae2a-445bb9af45d6-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn\" (UID: \"d589aff0-cdbd-43c1-ae2a-445bb9af45d6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn" Jul 26 18:29:40.817522 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.817502 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d589aff0-cdbd-43c1-ae2a-445bb9af45d6-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn\" (UID: \"d589aff0-cdbd-43c1-ae2a-445bb9af45d6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn" Jul 26 18:29:40.825534 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.825506 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d794n\" (UniqueName: \"kubernetes.io/projected/d589aff0-cdbd-43c1-ae2a-445bb9af45d6-kube-api-access-d794n\") pod \"payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn\" (UID: \"d589aff0-cdbd-43c1-ae2a-445bb9af45d6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn" Jul 26 18:29:40.884065 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.884026 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26" Jul 26 18:29:40.903392 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:40.903361 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn" Jul 26 18:29:41.040863 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:41.040831 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26"] Jul 26 18:29:41.042055 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:29:41.042027 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6594d7c_e383_4b97_bb10_2e5a61bb94d5.slice/crio-7a360a2e4d44bd3ca0b647a2ce6e3458ad488df8a280a910a2aa7f6ea7423d08 WatchSource:0}: Error finding container 7a360a2e4d44bd3ca0b647a2ce6e3458ad488df8a280a910a2aa7f6ea7423d08: Status 404 returned error can't find the container with id 7a360a2e4d44bd3ca0b647a2ce6e3458ad488df8a280a910a2aa7f6ea7423d08 Jul 26 18:29:41.068454 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:41.068431 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn"] Jul 26 18:29:41.074771 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:29:41.074731 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd589aff0_cdbd_43c1_ae2a_445bb9af45d6.slice/crio-d5b2cbf7a1a030d578d8fed54195410e4d54d27808efeac22581fc5cbc55a77b WatchSource:0}: Error finding container d5b2cbf7a1a030d578d8fed54195410e4d54d27808efeac22581fc5cbc55a77b: Status 404 returned error can't find the container with id d5b2cbf7a1a030d578d8fed54195410e4d54d27808efeac22581fc5cbc55a77b Jul 26 18:29:41.931623 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:41.931582 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn" event={"ID":"d589aff0-cdbd-43c1-ae2a-445bb9af45d6","Type":"ContainerStarted","Data":"950ae0608eb6a05b23e331a070a70df534826ed6fea40e375919d18f8b14b79e"} Jul 26 18:29:41.932127 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:41.931652 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn" event={"ID":"d589aff0-cdbd-43c1-ae2a-445bb9af45d6","Type":"ContainerStarted","Data":"d5b2cbf7a1a030d578d8fed54195410e4d54d27808efeac22581fc5cbc55a77b"} Jul 26 18:29:41.932127 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:41.931712 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn" Jul 26 18:29:41.933593 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:41.933563 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26" event={"ID":"f6594d7c-e383-4b97-bb10-2e5a61bb94d5","Type":"ContainerStarted","Data":"2b3b92cf1ffa7da2e47ee0bd3e6b8e576e82118e4e65ce1654c2691c7c1b6678"} Jul 26 18:29:41.933727 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:41.933620 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26" event={"ID":"f6594d7c-e383-4b97-bb10-2e5a61bb94d5","Type":"ContainerStarted","Data":"7a360a2e4d44bd3ca0b647a2ce6e3458ad488df8a280a910a2aa7f6ea7423d08"} Jul 26 18:29:41.933727 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:41.933654 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26" Jul 26 18:29:41.947880 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:41.947824 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn" podStartSLOduration=1.947807778 podStartE2EDuration="1.947807778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:29:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:29:41.946560317 +0000 UTC m=+2435.899216031" watchObservedRunningTime="2026-07-26 18:29:41.947807778 +0000 UTC m=+2435.900463465" Jul 26 18:29:41.962152 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:41.962103 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26" podStartSLOduration=1.962090083 podStartE2EDuration="1.962090083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:29:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:29:41.961231244 +0000 UTC m=+2435.913886945" watchObservedRunningTime="2026-07-26 18:29:41.962090083 +0000 UTC m=+2435.914745770" Jul 26 18:29:44.067177 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:44.067140 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:44.895061 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:44.895032 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv" Jul 26 18:29:44.895505 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:44.895486 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt" Jul 26 18:29:52.941075 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:52.941043 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn" Jul 26 18:29:52.941488 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:52.941126 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26" Jul 26 18:29:57.320340 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:57.320306 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:29:57.375107 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:29:57.375070 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:30:04.043844 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:30:04.043808 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:30:06.646574 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:30:06.646541 2559 scope.go:117] "RemoveContainer" containerID="7688abeab8da0bbef29c75baa32fe7957c774158d743708bdbe7f993924691c8" Jul 26 18:30:53.938856 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:30:53.938820 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:31:10.755151 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:31:10.755049 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:31:10.855614 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:31:10.855576 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:33:01.481556 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:33:01.481528 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:34:06.693979 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:34:06.693908 2559 scope.go:117] "RemoveContainer" containerID="2f49ab434471a10c317ff229653848be39a3e4c15d8e97fc0ea6684b46bb98eb" Jul 26 18:35:05.121829 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:35:05.121785 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:35:05.924487 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:35:05.924452 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:35:23.202576 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:35:23.202538 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:36:17.617943 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:36:17.617895 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:36:49.202380 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:36:49.202341 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:37:33.809472 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:33.809435 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:37:34.698003 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:34.697965 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:37:47.203204 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.203166 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr"] Jul 26 18:37:47.207060 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.207033 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:47.210709 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.210666 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-57f93c\"/\"rate-test-model-57f93c-kserve-self-signed-certs\"" Jul 26 18:37:47.210855 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.210687 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-57f93c\"/\"default-dockercfg-4rlzp\"" Jul 26 18:37:47.210855 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.210692 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-57f93c\"/\"kube-root-ca.crt\"" Jul 26 18:37:47.210855 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.210850 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-57f93c\"/\"openshift-service-ca.crt\"" Jul 26 18:37:47.216125 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.216101 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr"] Jul 26 18:37:47.234130 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.234088 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:37:47.237121 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.237097 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e779ab7b-6aaf-4595-b914-ad23c33ecf65-tls-certs\") pod \"rate-test-model-57f93c-kserve-57f58bc54c-87fdr\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:47.237245 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.237134 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd8tm\" (UniqueName: \"kubernetes.io/projected/e779ab7b-6aaf-4595-b914-ad23c33ecf65-kube-api-access-qd8tm\") pod \"rate-test-model-57f93c-kserve-57f58bc54c-87fdr\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:47.237245 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.237203 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-tmp-dir\") pod \"rate-test-model-57f93c-kserve-57f58bc54c-87fdr\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:47.237245 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.237236 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-dshm\") pod \"rate-test-model-57f93c-kserve-57f58bc54c-87fdr\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:47.237419 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.237275 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-home\") pod \"rate-test-model-57f93c-kserve-57f58bc54c-87fdr\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:47.237419 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.237311 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-model-cache\") pod \"rate-test-model-57f93c-kserve-57f58bc54c-87fdr\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:47.338573 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.338541 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e779ab7b-6aaf-4595-b914-ad23c33ecf65-tls-certs\") pod \"rate-test-model-57f93c-kserve-57f58bc54c-87fdr\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:47.338573 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.338576 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd8tm\" (UniqueName: \"kubernetes.io/projected/e779ab7b-6aaf-4595-b914-ad23c33ecf65-kube-api-access-qd8tm\") pod \"rate-test-model-57f93c-kserve-57f58bc54c-87fdr\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:47.338878 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.338611 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-tmp-dir\") pod \"rate-test-model-57f93c-kserve-57f58bc54c-87fdr\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:47.338878 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.338711 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-dshm\") pod \"rate-test-model-57f93c-kserve-57f58bc54c-87fdr\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:47.338878 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.338758 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-home\") pod \"rate-test-model-57f93c-kserve-57f58bc54c-87fdr\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:47.338878 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.338807 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-model-cache\") pod \"rate-test-model-57f93c-kserve-57f58bc54c-87fdr\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:47.339089 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.338971 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-tmp-dir\") pod \"rate-test-model-57f93c-kserve-57f58bc54c-87fdr\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:47.339148 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.339097 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-home\") pod \"rate-test-model-57f93c-kserve-57f58bc54c-87fdr\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:47.339205 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.339179 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-model-cache\") pod \"rate-test-model-57f93c-kserve-57f58bc54c-87fdr\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:47.341085 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.341058 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-dshm\") pod \"rate-test-model-57f93c-kserve-57f58bc54c-87fdr\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:47.341276 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.341260 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e779ab7b-6aaf-4595-b914-ad23c33ecf65-tls-certs\") pod \"rate-test-model-57f93c-kserve-57f58bc54c-87fdr\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:47.346113 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.346090 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd8tm\" (UniqueName: \"kubernetes.io/projected/e779ab7b-6aaf-4595-b914-ad23c33ecf65-kube-api-access-qd8tm\") pod \"rate-test-model-57f93c-kserve-57f58bc54c-87fdr\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:47.518022 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.517974 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:47.653651 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:47.653616 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr"] Jul 26 18:37:47.657149 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:37:47.657124 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode779ab7b_6aaf_4595_b914_ad23c33ecf65.slice/crio-e64e2e2b2ad1fbf27234dd29bf1d7913823c396b0f6a96d204425c14964897cf WatchSource:0}: Error finding container e64e2e2b2ad1fbf27234dd29bf1d7913823c396b0f6a96d204425c14964897cf: Status 404 returned error can't find the container with id e64e2e2b2ad1fbf27234dd29bf1d7913823c396b0f6a96d204425c14964897cf Jul 26 18:37:48.150197 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:48.150160 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" event={"ID":"e779ab7b-6aaf-4595-b914-ad23c33ecf65","Type":"ContainerStarted","Data":"f956709bbbba1327203df4ea72cc210f59615bbd4e814d84b9bc487ba82c6145"} Jul 26 18:37:48.150394 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:48.150210 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" event={"ID":"e779ab7b-6aaf-4595-b914-ad23c33ecf65","Type":"ContainerStarted","Data":"e64e2e2b2ad1fbf27234dd29bf1d7913823c396b0f6a96d204425c14964897cf"} Jul 26 18:37:48.167948 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:48.167898 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" podStartSLOduration=1.167880905 podStartE2EDuration="1.167880905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:37:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:37:48.16701205 +0000 UTC m=+2922.119667771" watchObservedRunningTime="2026-07-26 18:37:48.167880905 +0000 UTC m=+2922.120536593" Jul 26 18:37:57.518505 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:57.518467 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:57.519065 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:57.518566 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:57.531177 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:57.531150 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:37:58.207201 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:37:58.207173 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:38:12.316711 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:38:12.316661 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:38:14.571282 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:38:14.571247 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:39:26.041937 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:26.041905 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr"] Jul 26 18:39:26.064595 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:26.064554 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:39:27.003281 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:27.003241 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:39:41.060726 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.060639 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" podUID="e779ab7b-6aaf-4595-b914-ad23c33ecf65" containerName="main" containerID="cri-o://f956709bbbba1327203df4ea72cc210f59615bbd4e814d84b9bc487ba82c6145" gracePeriod=45 Jul 26 18:39:41.311153 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.311090 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:39:41.373595 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.373569 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-tmp-dir\") pod \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " Jul 26 18:39:41.373815 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.373634 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-dshm\") pod \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " Jul 26 18:39:41.373815 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.373658 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-home\") pod \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " Jul 26 18:39:41.373815 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.373699 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-model-cache\") pod \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " Jul 26 18:39:41.373815 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.373742 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e779ab7b-6aaf-4595-b914-ad23c33ecf65-tls-certs\") pod \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " Jul 26 18:39:41.373815 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.373785 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qd8tm\" (UniqueName: \"kubernetes.io/projected/e779ab7b-6aaf-4595-b914-ad23c33ecf65-kube-api-access-qd8tm\") pod \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\" (UID: \"e779ab7b-6aaf-4595-b914-ad23c33ecf65\") " Jul 26 18:39:41.374081 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.373940 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e779ab7b-6aaf-4595-b914-ad23c33ecf65" (UID: "e779ab7b-6aaf-4595-b914-ad23c33ecf65"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:39:41.374081 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.373944 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-home" (OuterVolumeSpecName: "home") pod "e779ab7b-6aaf-4595-b914-ad23c33ecf65" (UID: "e779ab7b-6aaf-4595-b914-ad23c33ecf65"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:39:41.374081 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.374008 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-model-cache" (OuterVolumeSpecName: "model-cache") pod "e779ab7b-6aaf-4595-b914-ad23c33ecf65" (UID: "e779ab7b-6aaf-4595-b914-ad23c33ecf65"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:39:41.374081 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.374054 2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-tmp-dir\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:39:41.374081 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.374071 2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-home\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:39:41.375969 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.375941 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-dshm" (OuterVolumeSpecName: "dshm") pod "e779ab7b-6aaf-4595-b914-ad23c33ecf65" (UID: "e779ab7b-6aaf-4595-b914-ad23c33ecf65"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 26 18:39:41.376085 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.375969 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e779ab7b-6aaf-4595-b914-ad23c33ecf65-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e779ab7b-6aaf-4595-b914-ad23c33ecf65" (UID: "e779ab7b-6aaf-4595-b914-ad23c33ecf65"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 26 18:39:41.376085 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.376005 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e779ab7b-6aaf-4595-b914-ad23c33ecf65-kube-api-access-qd8tm" (OuterVolumeSpecName: "kube-api-access-qd8tm") pod "e779ab7b-6aaf-4595-b914-ad23c33ecf65" (UID: "e779ab7b-6aaf-4595-b914-ad23c33ecf65"). InnerVolumeSpecName "kube-api-access-qd8tm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:39:41.475069 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.475031 2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-dshm\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:39:41.475069 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.475060 2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e779ab7b-6aaf-4595-b914-ad23c33ecf65-model-cache\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:39:41.475069 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.475070 2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e779ab7b-6aaf-4595-b914-ad23c33ecf65-tls-certs\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:39:41.475396 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.475078 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qd8tm\" (UniqueName: \"kubernetes.io/projected/e779ab7b-6aaf-4595-b914-ad23c33ecf65-kube-api-access-qd8tm\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:39:41.691982 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.691893 2559 generic.go:356] "Generic (PLEG): container finished" podID="e779ab7b-6aaf-4595-b914-ad23c33ecf65" containerID="f956709bbbba1327203df4ea72cc210f59615bbd4e814d84b9bc487ba82c6145" exitCode=0 Jul 26 18:39:41.691982 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.691931 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" Jul 26 18:39:41.691982 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.691935 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" event={"ID":"e779ab7b-6aaf-4595-b914-ad23c33ecf65","Type":"ContainerDied","Data":"f956709bbbba1327203df4ea72cc210f59615bbd4e814d84b9bc487ba82c6145"} Jul 26 18:39:41.691982 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.691973 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr" event={"ID":"e779ab7b-6aaf-4595-b914-ad23c33ecf65","Type":"ContainerDied","Data":"e64e2e2b2ad1fbf27234dd29bf1d7913823c396b0f6a96d204425c14964897cf"} Jul 26 18:39:41.692285 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.691994 2559 scope.go:117] "RemoveContainer" containerID="f956709bbbba1327203df4ea72cc210f59615bbd4e814d84b9bc487ba82c6145" Jul 26 18:39:41.701456 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.701432 2559 scope.go:117] "RemoveContainer" containerID="f956709bbbba1327203df4ea72cc210f59615bbd4e814d84b9bc487ba82c6145" Jul 26 18:39:41.701776 ip-10-0-128-209 kubenswrapper[2559]: E0726 18:39:41.701758 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f956709bbbba1327203df4ea72cc210f59615bbd4e814d84b9bc487ba82c6145\": container with ID starting with f956709bbbba1327203df4ea72cc210f59615bbd4e814d84b9bc487ba82c6145 not found: ID does not exist" containerID="f956709bbbba1327203df4ea72cc210f59615bbd4e814d84b9bc487ba82c6145" Jul 26 18:39:41.701852 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.701785 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f956709bbbba1327203df4ea72cc210f59615bbd4e814d84b9bc487ba82c6145"} err="failed to get container status \"f956709bbbba1327203df4ea72cc210f59615bbd4e814d84b9bc487ba82c6145\": rpc error: code = NotFound desc = could not find container \"f956709bbbba1327203df4ea72cc210f59615bbd4e814d84b9bc487ba82c6145\": container with ID starting with f956709bbbba1327203df4ea72cc210f59615bbd4e814d84b9bc487ba82c6145 not found: ID does not exist" Jul 26 18:39:41.714835 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.714809 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr"] Jul 26 18:39:41.718645 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:41.718624 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-57f93c/rate-test-model-57f93c-kserve-57f58bc54c-87fdr"] Jul 26 18:39:42.439415 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:42.439378 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e779ab7b-6aaf-4595-b914-ad23c33ecf65" path="/var/lib/kubelet/pods/e779ab7b-6aaf-4595-b914-ad23c33ecf65/volumes" Jul 26 18:39:46.858130 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:46.858090 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:39:47.235617 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:47.235583 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:39:47.541330 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:47.541243 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:39:49.940659 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:49.940620 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:39:50.040393 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:50.040359 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:39:50.316338 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:50.316306 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:39:50.440542 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:50.440508 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:39:51.026780 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.026742 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:39:51.096445 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.096414 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds"] Jul 26 18:39:51.096829 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.096815 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e779ab7b-6aaf-4595-b914-ad23c33ecf65" containerName="main" Jul 26 18:39:51.096880 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.096832 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e779ab7b-6aaf-4595-b914-ad23c33ecf65" containerName="main" Jul 26 18:39:51.096919 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.096898 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e779ab7b-6aaf-4595-b914-ad23c33ecf65" containerName="main" Jul 26 18:39:51.099898 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.099880 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds" Jul 26 18:39:51.102793 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.102771 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-76b936\"" Jul 26 18:39:51.116237 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.116208 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds"] Jul 26 18:39:51.120789 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.120762 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:39:51.126958 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.126919 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk"] Jul 26 18:39:51.132480 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.132455 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk" Jul 26 18:39:51.135358 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.135337 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-76b936-dockercfg-982p8\"" Jul 26 18:39:51.142119 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.142092 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk"] Jul 26 18:39:51.153175 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.153148 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10eab379-287a-4515-94b4-bdaa580ba9f8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk\" (UID: \"10eab379-287a-4515-94b4-bdaa580ba9f8\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk" Jul 26 18:39:51.153633 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.153258 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn474\" (UniqueName: \"kubernetes.io/projected/10eab379-287a-4515-94b4-bdaa580ba9f8-kube-api-access-pn474\") pod \"payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk\" (UID: \"10eab379-287a-4515-94b4-bdaa580ba9f8\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk" Jul 26 18:39:51.153633 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.153288 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4803572b-de48-4248-9c41-157759c3f2f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds\" (UID: \"4803572b-de48-4248-9c41-157759c3f2f4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds" Jul 26 18:39:51.153633 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.153380 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qb8l\" (UniqueName: \"kubernetes.io/projected/4803572b-de48-4248-9c41-157759c3f2f4-kube-api-access-9qb8l\") pod \"payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds\" (UID: \"4803572b-de48-4248-9c41-157759c3f2f4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds" Jul 26 18:39:51.254391 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.254352 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qb8l\" (UniqueName: \"kubernetes.io/projected/4803572b-de48-4248-9c41-157759c3f2f4-kube-api-access-9qb8l\") pod \"payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds\" (UID: \"4803572b-de48-4248-9c41-157759c3f2f4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds" Jul 26 18:39:51.254589 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.254478 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10eab379-287a-4515-94b4-bdaa580ba9f8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk\" (UID: \"10eab379-287a-4515-94b4-bdaa580ba9f8\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk" Jul 26 18:39:51.254589 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.254535 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn474\" (UniqueName: \"kubernetes.io/projected/10eab379-287a-4515-94b4-bdaa580ba9f8-kube-api-access-pn474\") pod \"payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk\" (UID: \"10eab379-287a-4515-94b4-bdaa580ba9f8\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk" Jul 26 18:39:51.254589 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.254557 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4803572b-de48-4248-9c41-157759c3f2f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds\" (UID: \"4803572b-de48-4248-9c41-157759c3f2f4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds" Jul 26 18:39:51.255336 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.255314 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4803572b-de48-4248-9c41-157759c3f2f4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds\" (UID: \"4803572b-de48-4248-9c41-157759c3f2f4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds" Jul 26 18:39:51.255517 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.255499 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10eab379-287a-4515-94b4-bdaa580ba9f8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk\" (UID: \"10eab379-287a-4515-94b4-bdaa580ba9f8\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk" Jul 26 18:39:51.263182 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.263157 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn474\" (UniqueName: \"kubernetes.io/projected/10eab379-287a-4515-94b4-bdaa580ba9f8-kube-api-access-pn474\") pod \"payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk\" (UID: \"10eab379-287a-4515-94b4-bdaa580ba9f8\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk" Jul 26 18:39:51.263398 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.263379 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qb8l\" (UniqueName: \"kubernetes.io/projected/4803572b-de48-4248-9c41-157759c3f2f4-kube-api-access-9qb8l\") pod \"payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds\" (UID: \"4803572b-de48-4248-9c41-157759c3f2f4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds" Jul 26 18:39:51.409849 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.409746 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds" Jul 26 18:39:51.444294 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.444262 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk" Jul 26 18:39:51.564710 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.564535 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds"] Jul 26 18:39:51.567512 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:39:51.567477 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4803572b_de48_4248_9c41_157759c3f2f4.slice/crio-43c4fd140ccd4e6a3210d80e2e094eccaf99b58fdadd234479af7d819a4b40ee WatchSource:0}: Error finding container 43c4fd140ccd4e6a3210d80e2e094eccaf99b58fdadd234479af7d819a4b40ee: Status 404 returned error can't find the container with id 43c4fd140ccd4e6a3210d80e2e094eccaf99b58fdadd234479af7d819a4b40ee Jul 26 18:39:51.604495 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.604469 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk"] Jul 26 18:39:51.606185 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:39:51.606160 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10eab379_287a_4515_94b4_bdaa580ba9f8.slice/crio-001e1bf6b32c9545344defd05e7f29a0dc3559ac5cd4d73c95a15d2a24fc9824 WatchSource:0}: Error finding container 001e1bf6b32c9545344defd05e7f29a0dc3559ac5cd4d73c95a15d2a24fc9824: Status 404 returned error can't find the container with id 001e1bf6b32c9545344defd05e7f29a0dc3559ac5cd4d73c95a15d2a24fc9824 Jul 26 18:39:51.740181 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.740145 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk" event={"ID":"10eab379-287a-4515-94b4-bdaa580ba9f8","Type":"ContainerStarted","Data":"ea5a1ea5db6e133666ff766526aa43c4f5563f4b4a6b7f0e7c35399c76a16ad9"} Jul 26 18:39:51.740723 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.740210 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk" event={"ID":"10eab379-287a-4515-94b4-bdaa580ba9f8","Type":"ContainerStarted","Data":"001e1bf6b32c9545344defd05e7f29a0dc3559ac5cd4d73c95a15d2a24fc9824"} Jul 26 18:39:51.740723 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.740242 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk" Jul 26 18:39:51.741871 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.741844 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds" event={"ID":"4803572b-de48-4248-9c41-157759c3f2f4","Type":"ContainerStarted","Data":"27724d0686bde22f5e6193933f0bc3af4d658a573a0775ec61f7dee1888413fa"} Jul 26 18:39:51.741997 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.741894 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds" event={"ID":"4803572b-de48-4248-9c41-157759c3f2f4","Type":"ContainerStarted","Data":"43c4fd140ccd4e6a3210d80e2e094eccaf99b58fdadd234479af7d819a4b40ee"} Jul 26 18:39:51.741997 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.741932 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds" Jul 26 18:39:51.764242 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.764192 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk" podStartSLOduration=0.764175867 podStartE2EDuration="764.175867ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:39:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:39:51.763421909 +0000 UTC m=+3045.716077598" watchObservedRunningTime="2026-07-26 18:39:51.764175867 +0000 UTC m=+3045.716831533" Jul 26 18:39:51.781085 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:51.781043 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds" podStartSLOduration=0.781029248 podStartE2EDuration="781.029248ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:39:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:39:51.779818403 +0000 UTC m=+3045.732474101" watchObservedRunningTime="2026-07-26 18:39:51.781029248 +0000 UTC m=+3045.733684936" Jul 26 18:39:54.281380 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:39:54.281340 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:40:02.749238 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:02.749150 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk" Jul 26 18:40:02.749238 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:02.749234 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds" Jul 26 18:40:03.862356 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:03.862303 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:40:03.928150 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:03.928103 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:40:05.171905 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:05.171872 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:40:05.468828 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:05.468786 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:40:05.670441 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:05.670409 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:40:08.920762 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:08.920721 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:40:09.010229 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.010192 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:40:09.084836 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.084795 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:40:09.375999 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.375965 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:40:09.615353 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.615317 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:40:09.721465 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.721429 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45"] Jul 26 18:40:09.726354 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.726325 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45" Jul 26 18:40:09.726708 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.726663 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:40:09.729914 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.729892 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-3e8329\"" Jul 26 18:40:09.730002 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.729989 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-3e8329-dockercfg-f5ccd\"" Jul 26 18:40:09.737477 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.737436 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45"] Jul 26 18:40:09.757133 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.757103 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp"] Jul 26 18:40:09.760433 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.760416 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp" Jul 26 18:40:09.770696 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.770653 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp"] Jul 26 18:40:09.825644 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.825608 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8t9n\" (UniqueName: \"kubernetes.io/projected/092c5c3d-c634-43c2-875b-5f7457bf3205-kube-api-access-h8t9n\") pod \"payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45\" (UID: \"092c5c3d-c634-43c2-875b-5f7457bf3205\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45" Jul 26 18:40:09.825841 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.825741 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdc95c03-788c-4cfd-8f23-a507331bec41-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp\" (UID: \"fdc95c03-788c-4cfd-8f23-a507331bec41\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp" Jul 26 18:40:09.825841 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.825809 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/092c5c3d-c634-43c2-875b-5f7457bf3205-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45\" (UID: \"092c5c3d-c634-43c2-875b-5f7457bf3205\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45" Jul 26 18:40:09.825841 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.825835 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngdvc\" (UniqueName: \"kubernetes.io/projected/fdc95c03-788c-4cfd-8f23-a507331bec41-kube-api-access-ngdvc\") pod \"payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp\" (UID: \"fdc95c03-788c-4cfd-8f23-a507331bec41\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp" Jul 26 18:40:09.926940 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.926900 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdc95c03-788c-4cfd-8f23-a507331bec41-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp\" (UID: \"fdc95c03-788c-4cfd-8f23-a507331bec41\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp" Jul 26 18:40:09.927423 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.926984 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/092c5c3d-c634-43c2-875b-5f7457bf3205-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45\" (UID: \"092c5c3d-c634-43c2-875b-5f7457bf3205\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45" Jul 26 18:40:09.927423 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.927013 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngdvc\" (UniqueName: \"kubernetes.io/projected/fdc95c03-788c-4cfd-8f23-a507331bec41-kube-api-access-ngdvc\") pod \"payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp\" (UID: \"fdc95c03-788c-4cfd-8f23-a507331bec41\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp" Jul 26 18:40:09.927423 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.927073 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8t9n\" (UniqueName: \"kubernetes.io/projected/092c5c3d-c634-43c2-875b-5f7457bf3205-kube-api-access-h8t9n\") pod \"payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45\" (UID: \"092c5c3d-c634-43c2-875b-5f7457bf3205\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45" Jul 26 18:40:09.927931 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.927909 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdc95c03-788c-4cfd-8f23-a507331bec41-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp\" (UID: \"fdc95c03-788c-4cfd-8f23-a507331bec41\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp" Jul 26 18:40:09.927981 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.927913 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/092c5c3d-c634-43c2-875b-5f7457bf3205-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45\" (UID: \"092c5c3d-c634-43c2-875b-5f7457bf3205\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45" Jul 26 18:40:09.935510 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.935477 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngdvc\" (UniqueName: \"kubernetes.io/projected/fdc95c03-788c-4cfd-8f23-a507331bec41-kube-api-access-ngdvc\") pod \"payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp\" (UID: \"fdc95c03-788c-4cfd-8f23-a507331bec41\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp" Jul 26 18:40:09.935710 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:09.935611 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8t9n\" (UniqueName: \"kubernetes.io/projected/092c5c3d-c634-43c2-875b-5f7457bf3205-kube-api-access-h8t9n\") pod \"payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45\" (UID: \"092c5c3d-c634-43c2-875b-5f7457bf3205\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45" Jul 26 18:40:10.037344 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:10.037256 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45" Jul 26 18:40:10.070585 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:10.070552 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp" Jul 26 18:40:10.228532 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:10.228507 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp"] Jul 26 18:40:10.231039 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:40:10.231013 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdc95c03_788c_4cfd_8f23_a507331bec41.slice/crio-bba82206f8a1360bd1ba2161fcd60bb215ae6f5f9d58b658c80a71395c086f8e WatchSource:0}: Error finding container bba82206f8a1360bd1ba2161fcd60bb215ae6f5f9d58b658c80a71395c086f8e: Status 404 returned error can't find the container with id bba82206f8a1360bd1ba2161fcd60bb215ae6f5f9d58b658c80a71395c086f8e Jul 26 18:40:10.391548 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:10.391525 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45"] Jul 26 18:40:10.393378 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:40:10.393355 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod092c5c3d_c634_43c2_875b_5f7457bf3205.slice/crio-5285e5d90850d8aa6e2d283e0c3556266134ff26a61a8a9cf5d0bce16baa2a1e WatchSource:0}: Error finding container 5285e5d90850d8aa6e2d283e0c3556266134ff26a61a8a9cf5d0bce16baa2a1e: Status 404 returned error can't find the container with id 5285e5d90850d8aa6e2d283e0c3556266134ff26a61a8a9cf5d0bce16baa2a1e Jul 26 18:40:10.834162 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:10.834114 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp" event={"ID":"fdc95c03-788c-4cfd-8f23-a507331bec41","Type":"ContainerStarted","Data":"df8cf0745028e7e4149bab7f3e65d859e300ab517a26038e434a1fc1f49cbc38"} Jul 26 18:40:10.834341 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:10.834216 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp" event={"ID":"fdc95c03-788c-4cfd-8f23-a507331bec41","Type":"ContainerStarted","Data":"bba82206f8a1360bd1ba2161fcd60bb215ae6f5f9d58b658c80a71395c086f8e"} Jul 26 18:40:10.834341 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:10.834258 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp" Jul 26 18:40:10.835730 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:10.835699 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45" event={"ID":"092c5c3d-c634-43c2-875b-5f7457bf3205","Type":"ContainerStarted","Data":"a858c6b8cff9c72866000595bbe0bb3740d73bc13819b0ed5767e823819b4825"} Jul 26 18:40:10.835847 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:10.835752 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45" event={"ID":"092c5c3d-c634-43c2-875b-5f7457bf3205","Type":"ContainerStarted","Data":"5285e5d90850d8aa6e2d283e0c3556266134ff26a61a8a9cf5d0bce16baa2a1e"} Jul 26 18:40:10.835847 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:10.835802 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45" Jul 26 18:40:10.855875 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:10.855827 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp" podStartSLOduration=1.855811192 podStartE2EDuration="1.855811192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:40:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:40:10.854075139 +0000 UTC m=+3064.806730844" watchObservedRunningTime="2026-07-26 18:40:10.855811192 +0000 UTC m=+3064.808466879" Jul 26 18:40:10.887628 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:10.887574 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45" podStartSLOduration=1.8875582020000001 podStartE2EDuration="1.887558202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:40:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:40:10.886164419 +0000 UTC m=+3064.838820109" watchObservedRunningTime="2026-07-26 18:40:10.887558202 +0000 UTC m=+3064.840213890" Jul 26 18:40:13.424812 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:13.424780 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:40:21.842752 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:21.842717 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp" Jul 26 18:40:21.843343 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:21.842798 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45" Jul 26 18:40:30.622900 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:30.622863 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp_fdc95c03-788c-4cfd-8f23-a507331bec41/payload-processing/0.log" Jul 26 18:40:45.300040 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:45.299999 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:40:49.596059 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:40:49.596024 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:41:27.699309 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:27.699276 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk_10eab379-287a-4515-94b4-bdaa580ba9f8/payload-processing/0.log" Jul 26 18:41:27.931017 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:27.930987 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp_fdc95c03-788c-4cfd-8f23-a507331bec41/payload-processing/0.log" Jul 26 18:41:28.205043 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:28.204952 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:41:28.533394 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:28.533357 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:41:28.726426 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:28.726384 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:41:33.699371 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:33.699332 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:41:33.779796 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:33.779749 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:41:33.851023 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:33.850983 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:41:34.187629 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.187593 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:41:34.364397 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.364363 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:41:34.464470 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.464436 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:41:34.472112 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.472080 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42"] Jul 26 18:41:34.475709 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.475662 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42" Jul 26 18:41:34.478398 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.478378 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-4147f2-dockercfg-cs5kj\"" Jul 26 18:41:34.478607 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.478595 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-4147f2\"" Jul 26 18:41:34.487795 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.487767 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42"] Jul 26 18:41:34.511019 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.510986 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s"] Jul 26 18:41:34.514455 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.514437 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s" Jul 26 18:41:34.524896 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.524863 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s"] Jul 26 18:41:34.581218 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.581183 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/083d7d91-1568-4f92-a707-53ebd763fdb9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s\" (UID: \"083d7d91-1568-4f92-a707-53ebd763fdb9\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s" Jul 26 18:41:34.581392 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.581254 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ec20e9f-b70c-4b2c-83bd-ad670ea41d72-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42\" (UID: \"3ec20e9f-b70c-4b2c-83bd-ad670ea41d72\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42" Jul 26 18:41:34.581392 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.581318 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssxtn\" (UniqueName: \"kubernetes.io/projected/3ec20e9f-b70c-4b2c-83bd-ad670ea41d72-kube-api-access-ssxtn\") pod \"payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42\" (UID: \"3ec20e9f-b70c-4b2c-83bd-ad670ea41d72\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42" Jul 26 18:41:34.581476 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.581397 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgfzn\" (UniqueName: \"kubernetes.io/projected/083d7d91-1568-4f92-a707-53ebd763fdb9-kube-api-access-wgfzn\") pod \"payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s\" (UID: \"083d7d91-1568-4f92-a707-53ebd763fdb9\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s" Jul 26 18:41:34.682867 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.682834 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ec20e9f-b70c-4b2c-83bd-ad670ea41d72-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42\" (UID: \"3ec20e9f-b70c-4b2c-83bd-ad670ea41d72\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42" Jul 26 18:41:34.683066 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.682889 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssxtn\" (UniqueName: \"kubernetes.io/projected/3ec20e9f-b70c-4b2c-83bd-ad670ea41d72-kube-api-access-ssxtn\") pod \"payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42\" (UID: \"3ec20e9f-b70c-4b2c-83bd-ad670ea41d72\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42" Jul 26 18:41:34.683066 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.682919 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgfzn\" (UniqueName: \"kubernetes.io/projected/083d7d91-1568-4f92-a707-53ebd763fdb9-kube-api-access-wgfzn\") pod \"payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s\" (UID: \"083d7d91-1568-4f92-a707-53ebd763fdb9\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s" Jul 26 18:41:34.683066 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.683010 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/083d7d91-1568-4f92-a707-53ebd763fdb9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s\" (UID: \"083d7d91-1568-4f92-a707-53ebd763fdb9\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s" Jul 26 18:41:34.683546 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.683525 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ec20e9f-b70c-4b2c-83bd-ad670ea41d72-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42\" (UID: \"3ec20e9f-b70c-4b2c-83bd-ad670ea41d72\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42" Jul 26 18:41:34.683700 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.683664 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/083d7d91-1568-4f92-a707-53ebd763fdb9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s\" (UID: \"083d7d91-1568-4f92-a707-53ebd763fdb9\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s" Jul 26 18:41:34.691146 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.691125 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgfzn\" (UniqueName: \"kubernetes.io/projected/083d7d91-1568-4f92-a707-53ebd763fdb9-kube-api-access-wgfzn\") pod \"payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s\" (UID: \"083d7d91-1568-4f92-a707-53ebd763fdb9\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s" Jul 26 18:41:34.691648 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.691626 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssxtn\" (UniqueName: \"kubernetes.io/projected/3ec20e9f-b70c-4b2c-83bd-ad670ea41d72-kube-api-access-ssxtn\") pod \"payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42\" (UID: \"3ec20e9f-b70c-4b2c-83bd-ad670ea41d72\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42" Jul 26 18:41:34.786528 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.786433 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42" Jul 26 18:41:34.827483 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:34.827452 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s" Jul 26 18:41:35.000298 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:35.000271 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s"] Jul 26 18:41:35.000607 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:41:35.000579 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod083d7d91_1568_4f92_a707_53ebd763fdb9.slice/crio-ee50764666ea1c04dd6c925e47ee25d0f41134ff85e987ca4f3b156f70222a7d WatchSource:0}: Error finding container ee50764666ea1c04dd6c925e47ee25d0f41134ff85e987ca4f3b156f70222a7d: Status 404 returned error can't find the container with id ee50764666ea1c04dd6c925e47ee25d0f41134ff85e987ca4f3b156f70222a7d Jul 26 18:41:35.147549 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:35.147519 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42"] Jul 26 18:41:35.148961 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:41:35.148930 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ec20e9f_b70c_4b2c_83bd_ad670ea41d72.slice/crio-dc41e5e53f22e89a0a29a412a6d454c22e88e9e81d99f7f2e776314b9f9b0e96 WatchSource:0}: Error finding container dc41e5e53f22e89a0a29a412a6d454c22e88e9e81d99f7f2e776314b9f9b0e96: Status 404 returned error can't find the container with id dc41e5e53f22e89a0a29a412a6d454c22e88e9e81d99f7f2e776314b9f9b0e96 Jul 26 18:41:35.242992 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:35.242949 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s" event={"ID":"083d7d91-1568-4f92-a707-53ebd763fdb9","Type":"ContainerStarted","Data":"74bebc0d884a6adfe390f2aacd495f0a2d01ac0df505e0e2f4b7f1ffb5365334"} Jul 26 18:41:35.243181 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:35.243052 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s" Jul 26 18:41:35.243181 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:35.243091 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s" event={"ID":"083d7d91-1568-4f92-a707-53ebd763fdb9","Type":"ContainerStarted","Data":"ee50764666ea1c04dd6c925e47ee25d0f41134ff85e987ca4f3b156f70222a7d"} Jul 26 18:41:35.244781 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:35.244755 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42" event={"ID":"3ec20e9f-b70c-4b2c-83bd-ad670ea41d72","Type":"ContainerStarted","Data":"2d19f3f9c239850eef9c4275ef836ebe0066d560440568790a2c919b96fe1fda"} Jul 26 18:41:35.244901 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:35.244795 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42" event={"ID":"3ec20e9f-b70c-4b2c-83bd-ad670ea41d72","Type":"ContainerStarted","Data":"dc41e5e53f22e89a0a29a412a6d454c22e88e9e81d99f7f2e776314b9f9b0e96"} Jul 26 18:41:35.244901 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:35.244817 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42" Jul 26 18:41:35.277425 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:35.277371 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s" podStartSLOduration=1.2773518400000001 podStartE2EDuration="1.27735184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:41:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:41:35.262427955 +0000 UTC m=+3149.215083642" watchObservedRunningTime="2026-07-26 18:41:35.27735184 +0000 UTC m=+3149.230007529" Jul 26 18:41:35.279094 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:35.279043 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42" podStartSLOduration=1.279032964 podStartE2EDuration="1.279032964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:41:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:41:35.276590803 +0000 UTC m=+3149.229246512" watchObservedRunningTime="2026-07-26 18:41:35.279032964 +0000 UTC m=+3149.231688703" Jul 26 18:41:38.213526 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:38.213486 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:41:46.251743 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:46.251712 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s" Jul 26 18:41:46.252332 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:46.251912 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42" Jul 26 18:41:59.615278 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:59.615231 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:41:59.636801 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:59.636766 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s"] Jul 26 18:41:59.637074 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:59.637047 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s" podUID="083d7d91-1568-4f92-a707-53ebd763fdb9" containerName="payload-processing" containerID="cri-o://74bebc0d884a6adfe390f2aacd495f0a2d01ac0df505e0e2f4b7f1ffb5365334" gracePeriod=30 Jul 26 18:41:59.641956 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:59.641929 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42"] Jul 26 18:41:59.642334 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:59.642307 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42" podUID="3ec20e9f-b70c-4b2c-83bd-ad670ea41d72" containerName="payload-pre-processing" containerID="cri-o://2d19f3f9c239850eef9c4275ef836ebe0066d560440568790a2c919b96fe1fda" gracePeriod=30 Jul 26 18:41:59.716501 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:59.716468 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:41:59.771219 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:59.771190 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:41:59.938491 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:59.938466 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s" Jul 26 18:41:59.945466 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:41:59.945444 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42" Jul 26 18:42:00.121020 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.120982 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ec20e9f-b70c-4b2c-83bd-ad670ea41d72-plugins-config-volume\") pod \"3ec20e9f-b70c-4b2c-83bd-ad670ea41d72\" (UID: \"3ec20e9f-b70c-4b2c-83bd-ad670ea41d72\") " Jul 26 18:42:00.121211 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.121082 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgfzn\" (UniqueName: \"kubernetes.io/projected/083d7d91-1568-4f92-a707-53ebd763fdb9-kube-api-access-wgfzn\") pod \"083d7d91-1568-4f92-a707-53ebd763fdb9\" (UID: \"083d7d91-1568-4f92-a707-53ebd763fdb9\") " Jul 26 18:42:00.121211 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.121106 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/083d7d91-1568-4f92-a707-53ebd763fdb9-plugins-config-volume\") pod \"083d7d91-1568-4f92-a707-53ebd763fdb9\" (UID: \"083d7d91-1568-4f92-a707-53ebd763fdb9\") " Jul 26 18:42:00.121211 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.121124 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssxtn\" (UniqueName: \"kubernetes.io/projected/3ec20e9f-b70c-4b2c-83bd-ad670ea41d72-kube-api-access-ssxtn\") pod \"3ec20e9f-b70c-4b2c-83bd-ad670ea41d72\" (UID: \"3ec20e9f-b70c-4b2c-83bd-ad670ea41d72\") " Jul 26 18:42:00.121491 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.121463 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ec20e9f-b70c-4b2c-83bd-ad670ea41d72-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3ec20e9f-b70c-4b2c-83bd-ad670ea41d72" (UID: "3ec20e9f-b70c-4b2c-83bd-ad670ea41d72"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:42:00.121574 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.121505 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/083d7d91-1568-4f92-a707-53ebd763fdb9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "083d7d91-1568-4f92-a707-53ebd763fdb9" (UID: "083d7d91-1568-4f92-a707-53ebd763fdb9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:42:00.123400 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.123372 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ec20e9f-b70c-4b2c-83bd-ad670ea41d72-kube-api-access-ssxtn" (OuterVolumeSpecName: "kube-api-access-ssxtn") pod "3ec20e9f-b70c-4b2c-83bd-ad670ea41d72" (UID: "3ec20e9f-b70c-4b2c-83bd-ad670ea41d72"). InnerVolumeSpecName "kube-api-access-ssxtn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:42:00.123493 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.123423 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/083d7d91-1568-4f92-a707-53ebd763fdb9-kube-api-access-wgfzn" (OuterVolumeSpecName: "kube-api-access-wgfzn") pod "083d7d91-1568-4f92-a707-53ebd763fdb9" (UID: "083d7d91-1568-4f92-a707-53ebd763fdb9"). InnerVolumeSpecName "kube-api-access-wgfzn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:42:00.222516 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.222484 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ec20e9f-b70c-4b2c-83bd-ad670ea41d72-plugins-config-volume\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:42:00.222516 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.222512 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgfzn\" (UniqueName: \"kubernetes.io/projected/083d7d91-1568-4f92-a707-53ebd763fdb9-kube-api-access-wgfzn\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:42:00.222516 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.222525 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/083d7d91-1568-4f92-a707-53ebd763fdb9-plugins-config-volume\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:42:00.222793 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.222534 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssxtn\" (UniqueName: \"kubernetes.io/projected/3ec20e9f-b70c-4b2c-83bd-ad670ea41d72-kube-api-access-ssxtn\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:42:00.373072 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.373037 2559 generic.go:356] "Generic (PLEG): container finished" podID="083d7d91-1568-4f92-a707-53ebd763fdb9" containerID="74bebc0d884a6adfe390f2aacd495f0a2d01ac0df505e0e2f4b7f1ffb5365334" exitCode=0 Jul 26 18:42:00.373229 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.373087 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s" Jul 26 18:42:00.373229 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.373130 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s" event={"ID":"083d7d91-1568-4f92-a707-53ebd763fdb9","Type":"ContainerDied","Data":"74bebc0d884a6adfe390f2aacd495f0a2d01ac0df505e0e2f4b7f1ffb5365334"} Jul 26 18:42:00.373229 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.373189 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s" event={"ID":"083d7d91-1568-4f92-a707-53ebd763fdb9","Type":"ContainerDied","Data":"ee50764666ea1c04dd6c925e47ee25d0f41134ff85e987ca4f3b156f70222a7d"} Jul 26 18:42:00.373229 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.373215 2559 scope.go:117] "RemoveContainer" containerID="74bebc0d884a6adfe390f2aacd495f0a2d01ac0df505e0e2f4b7f1ffb5365334" Jul 26 18:42:00.374441 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.374418 2559 generic.go:356] "Generic (PLEG): container finished" podID="3ec20e9f-b70c-4b2c-83bd-ad670ea41d72" containerID="2d19f3f9c239850eef9c4275ef836ebe0066d560440568790a2c919b96fe1fda" exitCode=0 Jul 26 18:42:00.374536 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.374461 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42" Jul 26 18:42:00.374536 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.374508 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42" event={"ID":"3ec20e9f-b70c-4b2c-83bd-ad670ea41d72","Type":"ContainerDied","Data":"2d19f3f9c239850eef9c4275ef836ebe0066d560440568790a2c919b96fe1fda"} Jul 26 18:42:00.374627 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.374548 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42" event={"ID":"3ec20e9f-b70c-4b2c-83bd-ad670ea41d72","Type":"ContainerDied","Data":"dc41e5e53f22e89a0a29a412a6d454c22e88e9e81d99f7f2e776314b9f9b0e96"} Jul 26 18:42:00.384185 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.383831 2559 scope.go:117] "RemoveContainer" containerID="74bebc0d884a6adfe390f2aacd495f0a2d01ac0df505e0e2f4b7f1ffb5365334" Jul 26 18:42:00.385178 ip-10-0-128-209 kubenswrapper[2559]: E0726 18:42:00.385154 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74bebc0d884a6adfe390f2aacd495f0a2d01ac0df505e0e2f4b7f1ffb5365334\": container with ID starting with 74bebc0d884a6adfe390f2aacd495f0a2d01ac0df505e0e2f4b7f1ffb5365334 not found: ID does not exist" containerID="74bebc0d884a6adfe390f2aacd495f0a2d01ac0df505e0e2f4b7f1ffb5365334" Jul 26 18:42:00.385178 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.385184 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74bebc0d884a6adfe390f2aacd495f0a2d01ac0df505e0e2f4b7f1ffb5365334"} err="failed to get container status \"74bebc0d884a6adfe390f2aacd495f0a2d01ac0df505e0e2f4b7f1ffb5365334\": rpc error: code = NotFound desc = could not find container \"74bebc0d884a6adfe390f2aacd495f0a2d01ac0df505e0e2f4b7f1ffb5365334\": container with ID starting with 74bebc0d884a6adfe390f2aacd495f0a2d01ac0df505e0e2f4b7f1ffb5365334 not found: ID does not exist" Jul 26 18:42:00.385360 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.385202 2559 scope.go:117] "RemoveContainer" containerID="2d19f3f9c239850eef9c4275ef836ebe0066d560440568790a2c919b96fe1fda" Jul 26 18:42:00.393294 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.393277 2559 scope.go:117] "RemoveContainer" containerID="2d19f3f9c239850eef9c4275ef836ebe0066d560440568790a2c919b96fe1fda" Jul 26 18:42:00.393535 ip-10-0-128-209 kubenswrapper[2559]: E0726 18:42:00.393517 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d19f3f9c239850eef9c4275ef836ebe0066d560440568790a2c919b96fe1fda\": container with ID starting with 2d19f3f9c239850eef9c4275ef836ebe0066d560440568790a2c919b96fe1fda not found: ID does not exist" containerID="2d19f3f9c239850eef9c4275ef836ebe0066d560440568790a2c919b96fe1fda" Jul 26 18:42:00.393582 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.393542 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d19f3f9c239850eef9c4275ef836ebe0066d560440568790a2c919b96fe1fda"} err="failed to get container status \"2d19f3f9c239850eef9c4275ef836ebe0066d560440568790a2c919b96fe1fda\": rpc error: code = NotFound desc = could not find container \"2d19f3f9c239850eef9c4275ef836ebe0066d560440568790a2c919b96fe1fda\": container with ID starting with 2d19f3f9c239850eef9c4275ef836ebe0066d560440568790a2c919b96fe1fda not found: ID does not exist" Jul 26 18:42:00.400368 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.400347 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42"] Jul 26 18:42:00.404248 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.404226 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-4147f2-6bbc889d8d-crl42"] Jul 26 18:42:00.415551 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.415525 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s"] Jul 26 18:42:00.418721 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.418702 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4147f2-765fcc6464-85f9s"] Jul 26 18:42:00.439785 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.439760 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="083d7d91-1568-4f92-a707-53ebd763fdb9" path="/var/lib/kubelet/pods/083d7d91-1568-4f92-a707-53ebd763fdb9/volumes" Jul 26 18:42:00.440123 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:00.440110 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ec20e9f-b70c-4b2c-83bd-ad670ea41d72" path="/var/lib/kubelet/pods/3ec20e9f-b70c-4b2c-83bd-ad670ea41d72/volumes" Jul 26 18:42:04.176686 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:04.176647 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:42:04.552408 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:04.552372 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:42:10.499041 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:10.499010 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:42:10.599222 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:10.599187 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:42:10.766270 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:10.766174 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:42:10.981180 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:10.981147 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:42:29.802896 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:29.802845 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:42:29.831998 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:29.831965 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk"] Jul 26 18:42:29.832309 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:29.832267 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk" podUID="10eab379-287a-4515-94b4-bdaa580ba9f8" containerName="payload-processing" containerID="cri-o://ea5a1ea5db6e133666ff766526aa43c4f5563f4b4a6b7f0e7c35399c76a16ad9" gracePeriod=30 Jul 26 18:42:29.842538 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:29.842511 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds"] Jul 26 18:42:29.842992 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:29.842943 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds" podUID="4803572b-de48-4248-9c41-157759c3f2f4" containerName="payload-pre-processing" containerID="cri-o://27724d0686bde22f5e6193933f0bc3af4d658a573a0775ec61f7dee1888413fa" gracePeriod=30 Jul 26 18:42:29.875873 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:29.875819 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:42:29.939004 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:29.938967 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:42:30.100215 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.100189 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk" Jul 26 18:42:30.142311 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.142287 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds" Jul 26 18:42:30.171201 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.171173 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4803572b-de48-4248-9c41-157759c3f2f4-plugins-config-volume\") pod \"4803572b-de48-4248-9c41-157759c3f2f4\" (UID: \"4803572b-de48-4248-9c41-157759c3f2f4\") " Jul 26 18:42:30.171201 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.171212 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10eab379-287a-4515-94b4-bdaa580ba9f8-plugins-config-volume\") pod \"10eab379-287a-4515-94b4-bdaa580ba9f8\" (UID: \"10eab379-287a-4515-94b4-bdaa580ba9f8\") " Jul 26 18:42:30.171449 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.171250 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pn474\" (UniqueName: \"kubernetes.io/projected/10eab379-287a-4515-94b4-bdaa580ba9f8-kube-api-access-pn474\") pod \"10eab379-287a-4515-94b4-bdaa580ba9f8\" (UID: \"10eab379-287a-4515-94b4-bdaa580ba9f8\") " Jul 26 18:42:30.171449 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.171273 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qb8l\" (UniqueName: \"kubernetes.io/projected/4803572b-de48-4248-9c41-157759c3f2f4-kube-api-access-9qb8l\") pod \"4803572b-de48-4248-9c41-157759c3f2f4\" (UID: \"4803572b-de48-4248-9c41-157759c3f2f4\") " Jul 26 18:42:30.171654 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.171628 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4803572b-de48-4248-9c41-157759c3f2f4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4803572b-de48-4248-9c41-157759c3f2f4" (UID: "4803572b-de48-4248-9c41-157759c3f2f4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:42:30.171777 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.171645 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10eab379-287a-4515-94b4-bdaa580ba9f8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "10eab379-287a-4515-94b4-bdaa580ba9f8" (UID: "10eab379-287a-4515-94b4-bdaa580ba9f8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:42:30.173422 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.173400 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10eab379-287a-4515-94b4-bdaa580ba9f8-kube-api-access-pn474" (OuterVolumeSpecName: "kube-api-access-pn474") pod "10eab379-287a-4515-94b4-bdaa580ba9f8" (UID: "10eab379-287a-4515-94b4-bdaa580ba9f8"). InnerVolumeSpecName "kube-api-access-pn474". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:42:30.173506 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.173424 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4803572b-de48-4248-9c41-157759c3f2f4-kube-api-access-9qb8l" (OuterVolumeSpecName: "kube-api-access-9qb8l") pod "4803572b-de48-4248-9c41-157759c3f2f4" (UID: "4803572b-de48-4248-9c41-157759c3f2f4"). InnerVolumeSpecName "kube-api-access-9qb8l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:42:30.272993 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.272948 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4803572b-de48-4248-9c41-157759c3f2f4-plugins-config-volume\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:42:30.272993 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.272983 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10eab379-287a-4515-94b4-bdaa580ba9f8-plugins-config-volume\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:42:30.272993 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.272994 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pn474\" (UniqueName: \"kubernetes.io/projected/10eab379-287a-4515-94b4-bdaa580ba9f8-kube-api-access-pn474\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:42:30.272993 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.273003 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qb8l\" (UniqueName: \"kubernetes.io/projected/4803572b-de48-4248-9c41-157759c3f2f4-kube-api-access-9qb8l\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:42:30.519046 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.519006 2559 generic.go:356] "Generic (PLEG): container finished" podID="4803572b-de48-4248-9c41-157759c3f2f4" containerID="27724d0686bde22f5e6193933f0bc3af4d658a573a0775ec61f7dee1888413fa" exitCode=0 Jul 26 18:42:30.519222 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.519055 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds" Jul 26 18:42:30.519222 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.519101 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds" event={"ID":"4803572b-de48-4248-9c41-157759c3f2f4","Type":"ContainerDied","Data":"27724d0686bde22f5e6193933f0bc3af4d658a573a0775ec61f7dee1888413fa"} Jul 26 18:42:30.519222 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.519161 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds" event={"ID":"4803572b-de48-4248-9c41-157759c3f2f4","Type":"ContainerDied","Data":"43c4fd140ccd4e6a3210d80e2e094eccaf99b58fdadd234479af7d819a4b40ee"} Jul 26 18:42:30.519222 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.519194 2559 scope.go:117] "RemoveContainer" containerID="27724d0686bde22f5e6193933f0bc3af4d658a573a0775ec61f7dee1888413fa" Jul 26 18:42:30.520843 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.520822 2559 generic.go:356] "Generic (PLEG): container finished" podID="10eab379-287a-4515-94b4-bdaa580ba9f8" containerID="ea5a1ea5db6e133666ff766526aa43c4f5563f4b4a6b7f0e7c35399c76a16ad9" exitCode=0 Jul 26 18:42:30.520957 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.520856 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk" event={"ID":"10eab379-287a-4515-94b4-bdaa580ba9f8","Type":"ContainerDied","Data":"ea5a1ea5db6e133666ff766526aa43c4f5563f4b4a6b7f0e7c35399c76a16ad9"} Jul 26 18:42:30.520957 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.520877 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk" Jul 26 18:42:30.520957 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.520896 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk" event={"ID":"10eab379-287a-4515-94b4-bdaa580ba9f8","Type":"ContainerDied","Data":"001e1bf6b32c9545344defd05e7f29a0dc3559ac5cd4d73c95a15d2a24fc9824"} Jul 26 18:42:30.530833 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.530818 2559 scope.go:117] "RemoveContainer" containerID="27724d0686bde22f5e6193933f0bc3af4d658a573a0775ec61f7dee1888413fa" Jul 26 18:42:30.531249 ip-10-0-128-209 kubenswrapper[2559]: E0726 18:42:30.531226 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27724d0686bde22f5e6193933f0bc3af4d658a573a0775ec61f7dee1888413fa\": container with ID starting with 27724d0686bde22f5e6193933f0bc3af4d658a573a0775ec61f7dee1888413fa not found: ID does not exist" containerID="27724d0686bde22f5e6193933f0bc3af4d658a573a0775ec61f7dee1888413fa" Jul 26 18:42:30.531337 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.531252 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27724d0686bde22f5e6193933f0bc3af4d658a573a0775ec61f7dee1888413fa"} err="failed to get container status \"27724d0686bde22f5e6193933f0bc3af4d658a573a0775ec61f7dee1888413fa\": rpc error: code = NotFound desc = could not find container \"27724d0686bde22f5e6193933f0bc3af4d658a573a0775ec61f7dee1888413fa\": container with ID starting with 27724d0686bde22f5e6193933f0bc3af4d658a573a0775ec61f7dee1888413fa not found: ID does not exist" Jul 26 18:42:30.531337 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.531268 2559 scope.go:117] "RemoveContainer" containerID="ea5a1ea5db6e133666ff766526aa43c4f5563f4b4a6b7f0e7c35399c76a16ad9" Jul 26 18:42:30.539106 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.539083 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds"] Jul 26 18:42:30.539412 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.539395 2559 scope.go:117] "RemoveContainer" containerID="ea5a1ea5db6e133666ff766526aa43c4f5563f4b4a6b7f0e7c35399c76a16ad9" Jul 26 18:42:30.539651 ip-10-0-128-209 kubenswrapper[2559]: E0726 18:42:30.539635 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea5a1ea5db6e133666ff766526aa43c4f5563f4b4a6b7f0e7c35399c76a16ad9\": container with ID starting with ea5a1ea5db6e133666ff766526aa43c4f5563f4b4a6b7f0e7c35399c76a16ad9 not found: ID does not exist" containerID="ea5a1ea5db6e133666ff766526aa43c4f5563f4b4a6b7f0e7c35399c76a16ad9" Jul 26 18:42:30.539775 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.539659 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea5a1ea5db6e133666ff766526aa43c4f5563f4b4a6b7f0e7c35399c76a16ad9"} err="failed to get container status \"ea5a1ea5db6e133666ff766526aa43c4f5563f4b4a6b7f0e7c35399c76a16ad9\": rpc error: code = NotFound desc = could not find container \"ea5a1ea5db6e133666ff766526aa43c4f5563f4b4a6b7f0e7c35399c76a16ad9\": container with ID starting with ea5a1ea5db6e133666ff766526aa43c4f5563f4b4a6b7f0e7c35399c76a16ad9 not found: ID does not exist" Jul 26 18:42:30.542548 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.542526 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-76b936-7fb4d4779b-gdfds"] Jul 26 18:42:30.553137 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.553111 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk"] Jul 26 18:42:30.559564 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:30.559542 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-76b936-6dd6c54748-lhpmk"] Jul 26 18:42:31.525266 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:31.525232 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:42:32.438722 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:32.438658 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10eab379-287a-4515-94b4-bdaa580ba9f8" path="/var/lib/kubelet/pods/10eab379-287a-4515-94b4-bdaa580ba9f8/volumes" Jul 26 18:42:32.439179 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:32.439163 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4803572b-de48-4248-9c41-157759c3f2f4" path="/var/lib/kubelet/pods/4803572b-de48-4248-9c41-157759c3f2f4/volumes" Jul 26 18:42:34.368623 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:34.368586 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:42:34.762074 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:34.762032 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:42:39.839337 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:42:39.839289 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:43:16.685876 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:16.685843 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:43:17.781840 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:17.781800 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:43:23.438112 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:23.438074 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:43:23.536562 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:23.536527 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:43:33.484975 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.484934 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:43:33.515317 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.515280 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp"] Jul 26 18:43:33.515555 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.515528 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp" podUID="fdc95c03-788c-4cfd-8f23-a507331bec41" containerName="payload-processing" containerID="cri-o://df8cf0745028e7e4149bab7f3e65d859e300ab517a26038e434a1fc1f49cbc38" gracePeriod=30 Jul 26 18:43:33.524599 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.524564 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45"] Jul 26 18:43:33.524857 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.524804 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45" podUID="092c5c3d-c634-43c2-875b-5f7457bf3205" containerName="payload-pre-processing" containerID="cri-o://a858c6b8cff9c72866000595bbe0bb3740d73bc13819b0ed5767e823819b4825" gracePeriod=30 Jul 26 18:43:33.556355 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.556317 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:43:33.628136 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.628101 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:43:33.794064 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.794035 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp" Jul 26 18:43:33.826658 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.826631 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45" Jul 26 18:43:33.832786 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.832742 2559 generic.go:356] "Generic (PLEG): container finished" podID="fdc95c03-788c-4cfd-8f23-a507331bec41" containerID="df8cf0745028e7e4149bab7f3e65d859e300ab517a26038e434a1fc1f49cbc38" exitCode=0 Jul 26 18:43:33.832786 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.832783 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp" Jul 26 18:43:33.833006 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.832888 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp" event={"ID":"fdc95c03-788c-4cfd-8f23-a507331bec41","Type":"ContainerDied","Data":"df8cf0745028e7e4149bab7f3e65d859e300ab517a26038e434a1fc1f49cbc38"} Jul 26 18:43:33.833006 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.832976 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp" event={"ID":"fdc95c03-788c-4cfd-8f23-a507331bec41","Type":"ContainerDied","Data":"bba82206f8a1360bd1ba2161fcd60bb215ae6f5f9d58b658c80a71395c086f8e"} Jul 26 18:43:33.833106 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.833040 2559 scope.go:117] "RemoveContainer" containerID="df8cf0745028e7e4149bab7f3e65d859e300ab517a26038e434a1fc1f49cbc38" Jul 26 18:43:33.834238 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.834185 2559 generic.go:356] "Generic (PLEG): container finished" podID="092c5c3d-c634-43c2-875b-5f7457bf3205" containerID="a858c6b8cff9c72866000595bbe0bb3740d73bc13819b0ed5767e823819b4825" exitCode=0 Jul 26 18:43:33.834238 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.834226 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45" Jul 26 18:43:33.834359 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.834228 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45" event={"ID":"092c5c3d-c634-43c2-875b-5f7457bf3205","Type":"ContainerDied","Data":"a858c6b8cff9c72866000595bbe0bb3740d73bc13819b0ed5767e823819b4825"} Jul 26 18:43:33.834359 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.834309 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45" event={"ID":"092c5c3d-c634-43c2-875b-5f7457bf3205","Type":"ContainerDied","Data":"5285e5d90850d8aa6e2d283e0c3556266134ff26a61a8a9cf5d0bce16baa2a1e"} Jul 26 18:43:33.841441 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.841421 2559 scope.go:117] "RemoveContainer" containerID="df8cf0745028e7e4149bab7f3e65d859e300ab517a26038e434a1fc1f49cbc38" Jul 26 18:43:33.841815 ip-10-0-128-209 kubenswrapper[2559]: E0726 18:43:33.841749 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df8cf0745028e7e4149bab7f3e65d859e300ab517a26038e434a1fc1f49cbc38\": container with ID starting with df8cf0745028e7e4149bab7f3e65d859e300ab517a26038e434a1fc1f49cbc38 not found: ID does not exist" containerID="df8cf0745028e7e4149bab7f3e65d859e300ab517a26038e434a1fc1f49cbc38" Jul 26 18:43:33.841815 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.841797 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df8cf0745028e7e4149bab7f3e65d859e300ab517a26038e434a1fc1f49cbc38"} err="failed to get container status \"df8cf0745028e7e4149bab7f3e65d859e300ab517a26038e434a1fc1f49cbc38\": rpc error: code = NotFound desc = could not find container \"df8cf0745028e7e4149bab7f3e65d859e300ab517a26038e434a1fc1f49cbc38\": container with ID starting with df8cf0745028e7e4149bab7f3e65d859e300ab517a26038e434a1fc1f49cbc38 not found: ID does not exist" Jul 26 18:43:33.841815 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.841816 2559 scope.go:117] "RemoveContainer" containerID="a858c6b8cff9c72866000595bbe0bb3740d73bc13819b0ed5767e823819b4825" Jul 26 18:43:33.851574 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.851554 2559 scope.go:117] "RemoveContainer" containerID="a858c6b8cff9c72866000595bbe0bb3740d73bc13819b0ed5767e823819b4825" Jul 26 18:43:33.851966 ip-10-0-128-209 kubenswrapper[2559]: E0726 18:43:33.851938 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a858c6b8cff9c72866000595bbe0bb3740d73bc13819b0ed5767e823819b4825\": container with ID starting with a858c6b8cff9c72866000595bbe0bb3740d73bc13819b0ed5767e823819b4825 not found: ID does not exist" containerID="a858c6b8cff9c72866000595bbe0bb3740d73bc13819b0ed5767e823819b4825" Jul 26 18:43:33.852043 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.851976 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a858c6b8cff9c72866000595bbe0bb3740d73bc13819b0ed5767e823819b4825"} err="failed to get container status \"a858c6b8cff9c72866000595bbe0bb3740d73bc13819b0ed5767e823819b4825\": rpc error: code = NotFound desc = could not find container \"a858c6b8cff9c72866000595bbe0bb3740d73bc13819b0ed5767e823819b4825\": container with ID starting with a858c6b8cff9c72866000595bbe0bb3740d73bc13819b0ed5767e823819b4825 not found: ID does not exist" Jul 26 18:43:33.954823 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.954789 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdc95c03-788c-4cfd-8f23-a507331bec41-plugins-config-volume\") pod \"fdc95c03-788c-4cfd-8f23-a507331bec41\" (UID: \"fdc95c03-788c-4cfd-8f23-a507331bec41\") " Jul 26 18:43:33.954823 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.954841 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/092c5c3d-c634-43c2-875b-5f7457bf3205-plugins-config-volume\") pod \"092c5c3d-c634-43c2-875b-5f7457bf3205\" (UID: \"092c5c3d-c634-43c2-875b-5f7457bf3205\") " Jul 26 18:43:33.955096 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.954882 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngdvc\" (UniqueName: \"kubernetes.io/projected/fdc95c03-788c-4cfd-8f23-a507331bec41-kube-api-access-ngdvc\") pod \"fdc95c03-788c-4cfd-8f23-a507331bec41\" (UID: \"fdc95c03-788c-4cfd-8f23-a507331bec41\") " Jul 26 18:43:33.955096 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.954931 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8t9n\" (UniqueName: \"kubernetes.io/projected/092c5c3d-c634-43c2-875b-5f7457bf3205-kube-api-access-h8t9n\") pod \"092c5c3d-c634-43c2-875b-5f7457bf3205\" (UID: \"092c5c3d-c634-43c2-875b-5f7457bf3205\") " Jul 26 18:43:33.955267 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.955241 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdc95c03-788c-4cfd-8f23-a507331bec41-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fdc95c03-788c-4cfd-8f23-a507331bec41" (UID: "fdc95c03-788c-4cfd-8f23-a507331bec41"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:43:33.955354 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.955333 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/092c5c3d-c634-43c2-875b-5f7457bf3205-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "092c5c3d-c634-43c2-875b-5f7457bf3205" (UID: "092c5c3d-c634-43c2-875b-5f7457bf3205"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:43:33.957154 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.957130 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdc95c03-788c-4cfd-8f23-a507331bec41-kube-api-access-ngdvc" (OuterVolumeSpecName: "kube-api-access-ngdvc") pod "fdc95c03-788c-4cfd-8f23-a507331bec41" (UID: "fdc95c03-788c-4cfd-8f23-a507331bec41"). InnerVolumeSpecName "kube-api-access-ngdvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:43:33.957259 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:33.957239 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/092c5c3d-c634-43c2-875b-5f7457bf3205-kube-api-access-h8t9n" (OuterVolumeSpecName: "kube-api-access-h8t9n") pod "092c5c3d-c634-43c2-875b-5f7457bf3205" (UID: "092c5c3d-c634-43c2-875b-5f7457bf3205"). InnerVolumeSpecName "kube-api-access-h8t9n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:43:34.056083 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:34.055985 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdc95c03-788c-4cfd-8f23-a507331bec41-plugins-config-volume\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:43:34.056083 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:34.056027 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/092c5c3d-c634-43c2-875b-5f7457bf3205-plugins-config-volume\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:43:34.056083 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:34.056038 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngdvc\" (UniqueName: \"kubernetes.io/projected/fdc95c03-788c-4cfd-8f23-a507331bec41-kube-api-access-ngdvc\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:43:34.056083 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:34.056046 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8t9n\" (UniqueName: \"kubernetes.io/projected/092c5c3d-c634-43c2-875b-5f7457bf3205-kube-api-access-h8t9n\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:43:34.156869 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:34.156835 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp"] Jul 26 18:43:34.160456 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:34.160433 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-3e8329-5d94fd75d4-j6hxp"] Jul 26 18:43:34.170778 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:34.170750 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45"] Jul 26 18:43:34.173851 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:34.173828 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-3e8329-7b5bd87c7b-bqw45"] Jul 26 18:43:34.438767 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:34.438666 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="092c5c3d-c634-43c2-875b-5f7457bf3205" path="/var/lib/kubelet/pods/092c5c3d-c634-43c2-875b-5f7457bf3205/volumes" Jul 26 18:43:34.439092 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:34.439078 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdc95c03-788c-4cfd-8f23-a507331bec41" path="/var/lib/kubelet/pods/fdc95c03-788c-4cfd-8f23-a507331bec41/volumes" Jul 26 18:43:37.948206 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:37.948164 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:43:38.002092 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:38.002054 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:43:38.417935 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:38.417888 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:43:44.063816 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:44.063764 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:43:44.164095 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:44.164057 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:43:44.332534 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:44.332438 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:43:44.539376 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:44.539337 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:43:55.196049 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.196013 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:43:55.271640 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.271605 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt"] Jul 26 18:43:55.271914 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.271886 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt" podUID="e500bed6-796b-408c-8a18-544c385e561e" containerName="payload-processing" containerID="cri-o://ecc5b6cfc2ff601ea683e4feadc1a08556f4626ba1c13034748da787f0859130" gracePeriod=30 Jul 26 18:43:55.277459 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.277429 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv"] Jul 26 18:43:55.277905 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.277658 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv" podUID="a573d182-06bd-4ee1-bce8-17ef92641caf" containerName="payload-pre-processing" containerID="cri-o://83b216af3b7efd0458163a363c1cbf9f9ab3cc5fa685d769232093aab874c5e0" gracePeriod=30 Jul 26 18:43:55.310289 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.310245 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:43:55.335929 ip-10-0-128-209 kubenswrapper[2559]: E0726 18:43:55.335888 2559 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda573d182_06bd_4ee1_bce8_17ef92641caf.slice/crio-conmon-83b216af3b7efd0458163a363c1cbf9f9ab3cc5fa685d769232093aab874c5e0.scope\": RecentStats: unable to find data in memory cache]" Jul 26 18:43:55.405216 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.405183 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:43:55.561389 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.559055 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt" Jul 26 18:43:55.565074 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.565045 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv" Jul 26 18:43:55.659143 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.659108 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9brl\" (UniqueName: \"kubernetes.io/projected/a573d182-06bd-4ee1-bce8-17ef92641caf-kube-api-access-w9brl\") pod \"a573d182-06bd-4ee1-bce8-17ef92641caf\" (UID: \"a573d182-06bd-4ee1-bce8-17ef92641caf\") " Jul 26 18:43:55.659334 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.659169 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrxdw\" (UniqueName: \"kubernetes.io/projected/e500bed6-796b-408c-8a18-544c385e561e-kube-api-access-jrxdw\") pod \"e500bed6-796b-408c-8a18-544c385e561e\" (UID: \"e500bed6-796b-408c-8a18-544c385e561e\") " Jul 26 18:43:55.659334 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.659189 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e500bed6-796b-408c-8a18-544c385e561e-plugins-config-volume\") pod \"e500bed6-796b-408c-8a18-544c385e561e\" (UID: \"e500bed6-796b-408c-8a18-544c385e561e\") " Jul 26 18:43:55.659334 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.659222 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a573d182-06bd-4ee1-bce8-17ef92641caf-plugins-config-volume\") pod \"a573d182-06bd-4ee1-bce8-17ef92641caf\" (UID: \"a573d182-06bd-4ee1-bce8-17ef92641caf\") " Jul 26 18:43:55.659598 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.659576 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e500bed6-796b-408c-8a18-544c385e561e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e500bed6-796b-408c-8a18-544c385e561e" (UID: "e500bed6-796b-408c-8a18-544c385e561e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:43:55.659661 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.659640 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a573d182-06bd-4ee1-bce8-17ef92641caf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a573d182-06bd-4ee1-bce8-17ef92641caf" (UID: "a573d182-06bd-4ee1-bce8-17ef92641caf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:43:55.661371 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.661353 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a573d182-06bd-4ee1-bce8-17ef92641caf-kube-api-access-w9brl" (OuterVolumeSpecName: "kube-api-access-w9brl") pod "a573d182-06bd-4ee1-bce8-17ef92641caf" (UID: "a573d182-06bd-4ee1-bce8-17ef92641caf"). InnerVolumeSpecName "kube-api-access-w9brl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:43:55.661434 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.661377 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e500bed6-796b-408c-8a18-544c385e561e-kube-api-access-jrxdw" (OuterVolumeSpecName: "kube-api-access-jrxdw") pod "e500bed6-796b-408c-8a18-544c385e561e" (UID: "e500bed6-796b-408c-8a18-544c385e561e"). InnerVolumeSpecName "kube-api-access-jrxdw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:43:55.760632 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.760542 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrxdw\" (UniqueName: \"kubernetes.io/projected/e500bed6-796b-408c-8a18-544c385e561e-kube-api-access-jrxdw\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:43:55.760632 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.760571 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e500bed6-796b-408c-8a18-544c385e561e-plugins-config-volume\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:43:55.760632 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.760583 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a573d182-06bd-4ee1-bce8-17ef92641caf-plugins-config-volume\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:43:55.760632 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.760592 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9brl\" (UniqueName: \"kubernetes.io/projected/a573d182-06bd-4ee1-bce8-17ef92641caf-kube-api-access-w9brl\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:43:55.938713 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.938653 2559 generic.go:356] "Generic (PLEG): container finished" podID="e500bed6-796b-408c-8a18-544c385e561e" containerID="ecc5b6cfc2ff601ea683e4feadc1a08556f4626ba1c13034748da787f0859130" exitCode=0 Jul 26 18:43:55.938713 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.938710 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt" Jul 26 18:43:55.938978 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.938749 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt" event={"ID":"e500bed6-796b-408c-8a18-544c385e561e","Type":"ContainerDied","Data":"ecc5b6cfc2ff601ea683e4feadc1a08556f4626ba1c13034748da787f0859130"} Jul 26 18:43:55.938978 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.938806 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt" event={"ID":"e500bed6-796b-408c-8a18-544c385e561e","Type":"ContainerDied","Data":"6860ce1074c384751e318420d7fdc7db05b1de2b5942bac10032d90c472e36b1"} Jul 26 18:43:55.938978 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.938838 2559 scope.go:117] "RemoveContainer" containerID="ecc5b6cfc2ff601ea683e4feadc1a08556f4626ba1c13034748da787f0859130" Jul 26 18:43:55.939958 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.939904 2559 generic.go:356] "Generic (PLEG): container finished" podID="a573d182-06bd-4ee1-bce8-17ef92641caf" containerID="83b216af3b7efd0458163a363c1cbf9f9ab3cc5fa685d769232093aab874c5e0" exitCode=0 Jul 26 18:43:55.939958 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.939929 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv" event={"ID":"a573d182-06bd-4ee1-bce8-17ef92641caf","Type":"ContainerDied","Data":"83b216af3b7efd0458163a363c1cbf9f9ab3cc5fa685d769232093aab874c5e0"} Jul 26 18:43:55.940094 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.939970 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv" Jul 26 18:43:55.940094 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.939977 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv" event={"ID":"a573d182-06bd-4ee1-bce8-17ef92641caf","Type":"ContainerDied","Data":"12d12564a18938c81cb20ca6c0f8feb59c1d7d0c4814f23794db1e9a2412261f"} Jul 26 18:43:55.948308 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.948289 2559 scope.go:117] "RemoveContainer" containerID="ecc5b6cfc2ff601ea683e4feadc1a08556f4626ba1c13034748da787f0859130" Jul 26 18:43:55.948566 ip-10-0-128-209 kubenswrapper[2559]: E0726 18:43:55.948546 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecc5b6cfc2ff601ea683e4feadc1a08556f4626ba1c13034748da787f0859130\": container with ID starting with ecc5b6cfc2ff601ea683e4feadc1a08556f4626ba1c13034748da787f0859130 not found: ID does not exist" containerID="ecc5b6cfc2ff601ea683e4feadc1a08556f4626ba1c13034748da787f0859130" Jul 26 18:43:55.948629 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.948597 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecc5b6cfc2ff601ea683e4feadc1a08556f4626ba1c13034748da787f0859130"} err="failed to get container status \"ecc5b6cfc2ff601ea683e4feadc1a08556f4626ba1c13034748da787f0859130\": rpc error: code = NotFound desc = could not find container \"ecc5b6cfc2ff601ea683e4feadc1a08556f4626ba1c13034748da787f0859130\": container with ID starting with ecc5b6cfc2ff601ea683e4feadc1a08556f4626ba1c13034748da787f0859130 not found: ID does not exist" Jul 26 18:43:55.948629 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.948622 2559 scope.go:117] "RemoveContainer" containerID="83b216af3b7efd0458163a363c1cbf9f9ab3cc5fa685d769232093aab874c5e0" Jul 26 18:43:55.956497 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.956478 2559 scope.go:117] "RemoveContainer" containerID="83b216af3b7efd0458163a363c1cbf9f9ab3cc5fa685d769232093aab874c5e0" Jul 26 18:43:55.956776 ip-10-0-128-209 kubenswrapper[2559]: E0726 18:43:55.956758 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83b216af3b7efd0458163a363c1cbf9f9ab3cc5fa685d769232093aab874c5e0\": container with ID starting with 83b216af3b7efd0458163a363c1cbf9f9ab3cc5fa685d769232093aab874c5e0 not found: ID does not exist" containerID="83b216af3b7efd0458163a363c1cbf9f9ab3cc5fa685d769232093aab874c5e0" Jul 26 18:43:55.956826 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.956784 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83b216af3b7efd0458163a363c1cbf9f9ab3cc5fa685d769232093aab874c5e0"} err="failed to get container status \"83b216af3b7efd0458163a363c1cbf9f9ab3cc5fa685d769232093aab874c5e0\": rpc error: code = NotFound desc = could not find container \"83b216af3b7efd0458163a363c1cbf9f9ab3cc5fa685d769232093aab874c5e0\": container with ID starting with 83b216af3b7efd0458163a363c1cbf9f9ab3cc5fa685d769232093aab874c5e0 not found: ID does not exist" Jul 26 18:43:55.961073 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.961051 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt"] Jul 26 18:43:55.964989 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.964964 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-57f93c-5bb578b65d-4vjzt"] Jul 26 18:43:55.974507 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.974483 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv"] Jul 26 18:43:55.977592 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:55.977567 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-57f93c-7f4cdd89bd-vpxpv"] Jul 26 18:43:56.439171 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:56.439141 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a573d182-06bd-4ee1-bce8-17ef92641caf" path="/var/lib/kubelet/pods/a573d182-06bd-4ee1-bce8-17ef92641caf/volumes" Jul 26 18:43:56.439553 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:56.439538 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e500bed6-796b-408c-8a18-544c385e561e" path="/var/lib/kubelet/pods/e500bed6-796b-408c-8a18-544c385e561e/volumes" Jul 26 18:43:59.749645 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:43:59.749611 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:44:00.122055 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:00.121947 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:44:05.884786 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:05.884746 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:44:05.950851 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:05.950806 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:44:06.151210 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:06.151107 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:44:06.379889 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:06.379854 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:44:16.921545 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:16.921504 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:44:16.950899 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:16.950866 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn"] Jul 26 18:44:16.951113 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:16.951069 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn" podUID="d589aff0-cdbd-43c1-ae2a-445bb9af45d6" containerName="payload-processing" containerID="cri-o://950ae0608eb6a05b23e331a070a70df534826ed6fea40e375919d18f8b14b79e" gracePeriod=30 Jul 26 18:44:16.958536 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:16.958499 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26"] Jul 26 18:44:16.959008 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:16.958796 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26" podUID="f6594d7c-e383-4b97-bb10-2e5a61bb94d5" containerName="payload-pre-processing" containerID="cri-o://2b3b92cf1ffa7da2e47ee0bd3e6b8e576e82118e4e65ce1654c2691c7c1b6678" gracePeriod=30 Jul 26 18:44:16.990621 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:16.990589 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:44:17.061953 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:17.061920 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:44:17.234808 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:17.234781 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn" Jul 26 18:44:17.238092 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:17.238061 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26" Jul 26 18:44:17.351221 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:17.351186 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d794n\" (UniqueName: \"kubernetes.io/projected/d589aff0-cdbd-43c1-ae2a-445bb9af45d6-kube-api-access-d794n\") pod \"d589aff0-cdbd-43c1-ae2a-445bb9af45d6\" (UID: \"d589aff0-cdbd-43c1-ae2a-445bb9af45d6\") " Jul 26 18:44:17.351392 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:17.351250 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6594d7c-e383-4b97-bb10-2e5a61bb94d5-plugins-config-volume\") pod \"f6594d7c-e383-4b97-bb10-2e5a61bb94d5\" (UID: \"f6594d7c-e383-4b97-bb10-2e5a61bb94d5\") " Jul 26 18:44:17.351392 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:17.351382 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trw2z\" (UniqueName: \"kubernetes.io/projected/f6594d7c-e383-4b97-bb10-2e5a61bb94d5-kube-api-access-trw2z\") pod \"f6594d7c-e383-4b97-bb10-2e5a61bb94d5\" (UID: \"f6594d7c-e383-4b97-bb10-2e5a61bb94d5\") " Jul 26 18:44:17.351460 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:17.351400 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d589aff0-cdbd-43c1-ae2a-445bb9af45d6-plugins-config-volume\") pod \"d589aff0-cdbd-43c1-ae2a-445bb9af45d6\" (UID: \"d589aff0-cdbd-43c1-ae2a-445bb9af45d6\") " Jul 26 18:44:17.351649 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:17.351623 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6594d7c-e383-4b97-bb10-2e5a61bb94d5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f6594d7c-e383-4b97-bb10-2e5a61bb94d5" (UID: "f6594d7c-e383-4b97-bb10-2e5a61bb94d5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:44:17.351849 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:17.351820 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d589aff0-cdbd-43c1-ae2a-445bb9af45d6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d589aff0-cdbd-43c1-ae2a-445bb9af45d6" (UID: "d589aff0-cdbd-43c1-ae2a-445bb9af45d6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 26 18:44:17.353359 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:17.353336 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d589aff0-cdbd-43c1-ae2a-445bb9af45d6-kube-api-access-d794n" (OuterVolumeSpecName: "kube-api-access-d794n") pod "d589aff0-cdbd-43c1-ae2a-445bb9af45d6" (UID: "d589aff0-cdbd-43c1-ae2a-445bb9af45d6"). InnerVolumeSpecName "kube-api-access-d794n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:44:17.353439 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:17.353358 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6594d7c-e383-4b97-bb10-2e5a61bb94d5-kube-api-access-trw2z" (OuterVolumeSpecName: "kube-api-access-trw2z") pod "f6594d7c-e383-4b97-bb10-2e5a61bb94d5" (UID: "f6594d7c-e383-4b97-bb10-2e5a61bb94d5"). InnerVolumeSpecName "kube-api-access-trw2z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 26 18:44:17.452608 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:17.452566 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6594d7c-e383-4b97-bb10-2e5a61bb94d5-plugins-config-volume\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:44:17.452608 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:17.452601 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trw2z\" (UniqueName: \"kubernetes.io/projected/f6594d7c-e383-4b97-bb10-2e5a61bb94d5-kube-api-access-trw2z\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:44:17.452608 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:17.452615 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d589aff0-cdbd-43c1-ae2a-445bb9af45d6-plugins-config-volume\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:44:17.452886 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:17.452628 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d794n\" (UniqueName: \"kubernetes.io/projected/d589aff0-cdbd-43c1-ae2a-445bb9af45d6-kube-api-access-d794n\") on node \"ip-10-0-128-209.ec2.internal\" DevicePath \"\"" Jul 26 18:44:18.044049 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:18.044018 2559 generic.go:356] "Generic (PLEG): container finished" podID="d589aff0-cdbd-43c1-ae2a-445bb9af45d6" containerID="950ae0608eb6a05b23e331a070a70df534826ed6fea40e375919d18f8b14b79e" exitCode=0 Jul 26 18:44:18.044521 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:18.044060 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn" Jul 26 18:44:18.044521 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:18.044094 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn" event={"ID":"d589aff0-cdbd-43c1-ae2a-445bb9af45d6","Type":"ContainerDied","Data":"950ae0608eb6a05b23e331a070a70df534826ed6fea40e375919d18f8b14b79e"} Jul 26 18:44:18.044521 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:18.044142 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn" event={"ID":"d589aff0-cdbd-43c1-ae2a-445bb9af45d6","Type":"ContainerDied","Data":"d5b2cbf7a1a030d578d8fed54195410e4d54d27808efeac22581fc5cbc55a77b"} Jul 26 18:44:18.044521 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:18.044168 2559 scope.go:117] "RemoveContainer" containerID="950ae0608eb6a05b23e331a070a70df534826ed6fea40e375919d18f8b14b79e" Jul 26 18:44:18.045623 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:18.045429 2559 generic.go:356] "Generic (PLEG): container finished" podID="f6594d7c-e383-4b97-bb10-2e5a61bb94d5" containerID="2b3b92cf1ffa7da2e47ee0bd3e6b8e576e82118e4e65ce1654c2691c7c1b6678" exitCode=0 Jul 26 18:44:18.045623 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:18.045464 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26" Jul 26 18:44:18.045623 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:18.045486 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26" event={"ID":"f6594d7c-e383-4b97-bb10-2e5a61bb94d5","Type":"ContainerDied","Data":"2b3b92cf1ffa7da2e47ee0bd3e6b8e576e82118e4e65ce1654c2691c7c1b6678"} Jul 26 18:44:18.045623 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:18.045532 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26" event={"ID":"f6594d7c-e383-4b97-bb10-2e5a61bb94d5","Type":"ContainerDied","Data":"7a360a2e4d44bd3ca0b647a2ce6e3458ad488df8a280a910a2aa7f6ea7423d08"} Jul 26 18:44:18.054048 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:18.053979 2559 scope.go:117] "RemoveContainer" containerID="950ae0608eb6a05b23e331a070a70df534826ed6fea40e375919d18f8b14b79e" Jul 26 18:44:18.054301 ip-10-0-128-209 kubenswrapper[2559]: E0726 18:44:18.054278 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"950ae0608eb6a05b23e331a070a70df534826ed6fea40e375919d18f8b14b79e\": container with ID starting with 950ae0608eb6a05b23e331a070a70df534826ed6fea40e375919d18f8b14b79e not found: ID does not exist" containerID="950ae0608eb6a05b23e331a070a70df534826ed6fea40e375919d18f8b14b79e" Jul 26 18:44:18.054360 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:18.054311 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"950ae0608eb6a05b23e331a070a70df534826ed6fea40e375919d18f8b14b79e"} err="failed to get container status \"950ae0608eb6a05b23e331a070a70df534826ed6fea40e375919d18f8b14b79e\": rpc error: code = NotFound desc = could not find container \"950ae0608eb6a05b23e331a070a70df534826ed6fea40e375919d18f8b14b79e\": container with ID starting with 950ae0608eb6a05b23e331a070a70df534826ed6fea40e375919d18f8b14b79e not found: ID does not exist" Jul 26 18:44:18.054360 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:18.054329 2559 scope.go:117] "RemoveContainer" containerID="2b3b92cf1ffa7da2e47ee0bd3e6b8e576e82118e4e65ce1654c2691c7c1b6678" Jul 26 18:44:18.062482 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:18.062463 2559 scope.go:117] "RemoveContainer" containerID="2b3b92cf1ffa7da2e47ee0bd3e6b8e576e82118e4e65ce1654c2691c7c1b6678" Jul 26 18:44:18.062765 ip-10-0-128-209 kubenswrapper[2559]: E0726 18:44:18.062734 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b3b92cf1ffa7da2e47ee0bd3e6b8e576e82118e4e65ce1654c2691c7c1b6678\": container with ID starting with 2b3b92cf1ffa7da2e47ee0bd3e6b8e576e82118e4e65ce1654c2691c7c1b6678 not found: ID does not exist" containerID="2b3b92cf1ffa7da2e47ee0bd3e6b8e576e82118e4e65ce1654c2691c7c1b6678" Jul 26 18:44:18.062851 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:18.062766 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b3b92cf1ffa7da2e47ee0bd3e6b8e576e82118e4e65ce1654c2691c7c1b6678"} err="failed to get container status \"2b3b92cf1ffa7da2e47ee0bd3e6b8e576e82118e4e65ce1654c2691c7c1b6678\": rpc error: code = NotFound desc = could not find container \"2b3b92cf1ffa7da2e47ee0bd3e6b8e576e82118e4e65ce1654c2691c7c1b6678\": container with ID starting with 2b3b92cf1ffa7da2e47ee0bd3e6b8e576e82118e4e65ce1654c2691c7c1b6678 not found: ID does not exist" Jul 26 18:44:18.068456 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:18.068432 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26"] Jul 26 18:44:18.071885 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:18.071864 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-6aa894-5f9764c99b-9vb26"] Jul 26 18:44:18.081619 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:18.081599 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn"] Jul 26 18:44:18.085425 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:18.085403 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-6aa894-7d6df7fb54-9b5fn"] Jul 26 18:44:18.440393 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:18.440314 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d589aff0-cdbd-43c1-ae2a-445bb9af45d6" path="/var/lib/kubelet/pods/d589aff0-cdbd-43c1-ae2a-445bb9af45d6/volumes" Jul 26 18:44:18.440725 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:18.440711 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6594d7c-e383-4b97-bb10-2e5a61bb94d5" path="/var/lib/kubelet/pods/f6594d7c-e383-4b97-bb10-2e5a61bb94d5/volumes" Jul 26 18:44:19.771108 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:19.771074 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:44:21.563897 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:21.563857 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:44:21.930427 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:21.930340 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:44:27.848129 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:27.848092 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:44:27.912203 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:27.912151 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:44:28.120318 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:28.120196 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:44:28.341607 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:28.341571 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:44:34.390326 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:34.390296 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67b95f7b6f-krcq4_2098ad1a-2857-4700-ab98-922266c839f5/manager/0.log" Jul 26 18:44:34.501970 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:34.501946 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-jjvgl_63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23/server/0.log" Jul 26 18:44:34.855229 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:34.855201 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b79b7df6c-rjcmd_2cec0f48-bf3f-4a86-b72d-602edc087625/copy-manifests/0.log" Jul 26 18:44:34.862515 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:34.862485 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b79b7df6c-rjcmd_2cec0f48-bf3f-4a86-b72d-602edc087625/manager/0.log" Jul 26 18:44:35.800061 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:35.800030 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-qtk7x_07edbc3f-69bc-4c38-ad7d-38b81abac5c9/postgres/0.log" Jul 26 18:44:37.177262 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:37.177236 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-t2h8t_1d8e314e-f2cb-4018-aa98-cff979e2fba8/manager/0.log" Jul 26 18:44:37.638284 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:37.638243 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-nxqxj_15d2d1c7-390b-450b-82e9-2d5b49a022f1/manager/0.log" Jul 26 18:44:38.348402 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:38.348367 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-xmcxd_f3875b31-ffe3-48dd-b240-0e799813a0ef/discovery/0.log" Jul 26 18:44:38.456737 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:38.456707 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6855b89bd5-2586g_31bcded3-9797-486d-b175-9ef1fc6d0b1a/kube-auth-proxy/0.log" Jul 26 18:44:39.495689 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:39.495650 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-p8x77_d9120c21-cdb3-42ae-b0f5-58b546a63710/main/0.log" Jul 26 18:44:39.722659 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:39.722630 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj54gq_eefed328-c33c-4529-ac0c-92415c3cafd5/main/0.log" Jul 26 18:44:39.832075 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:39.831996 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-lhjhb_5cbc9127-b44e-4a3d-9ebf-8414fa0aacf4/main/0.log" Jul 26 18:44:45.724569 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:45.724529 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:44:45.823683 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:45.823647 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:44:48.540995 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:48.540965 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fgwwx_92dc3c1b-48ce-4b80-858c-5421f0bcc2f2/global-pull-secret-syncer/0.log" Jul 26 18:44:48.608209 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:48.608178 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" Jul 26 18:44:48.724687 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:48.724645 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-209.ec2.internal_c79721d966a6282ec9f6039747948682/haproxy/0.log" Jul 26 18:44:54.196625 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:54.196593 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-t2h8t_1d8e314e-f2cb-4018-aa98-cff979e2fba8/manager/0.log" Jul 26 18:44:54.520173 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:54.520139 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-nxqxj_15d2d1c7-390b-450b-82e9-2d5b49a022f1/manager/0.log" Jul 26 18:44:56.598943 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:56.598904 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-92gfp_24a734d6-b903-4cfc-b9dd-d98bb9c336f3/cluster-monitoring-operator/0.log" Jul 26 18:44:56.740701 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:56.740656 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5mk9m_324bf551-2df7-4565-a667-33e4f050e53a/node-exporter/0.log" Jul 26 18:44:56.760370 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:56.760342 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5mk9m_324bf551-2df7-4565-a667-33e4f050e53a/kube-rbac-proxy/0.log" Jul 26 18:44:56.779399 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:56.779370 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5mk9m_324bf551-2df7-4565-a667-33e4f050e53a/init-textfile/0.log" Jul 26 18:44:58.325339 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325304 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5"] Jul 26 18:44:58.325759 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325616 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a573d182-06bd-4ee1-bce8-17ef92641caf" containerName="payload-pre-processing" Jul 26 18:44:58.325759 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325626 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a573d182-06bd-4ee1-bce8-17ef92641caf" containerName="payload-pre-processing" Jul 26 18:44:58.325759 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325636 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d589aff0-cdbd-43c1-ae2a-445bb9af45d6" containerName="payload-processing" Jul 26 18:44:58.325759 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325641 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d589aff0-cdbd-43c1-ae2a-445bb9af45d6" containerName="payload-processing" Jul 26 18:44:58.325759 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325650 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4803572b-de48-4248-9c41-157759c3f2f4" containerName="payload-pre-processing" Jul 26 18:44:58.325759 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325657 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4803572b-de48-4248-9c41-157759c3f2f4" containerName="payload-pre-processing" Jul 26 18:44:58.325759 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325663 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="083d7d91-1568-4f92-a707-53ebd763fdb9" containerName="payload-processing" Jul 26 18:44:58.325759 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325682 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="083d7d91-1568-4f92-a707-53ebd763fdb9" containerName="payload-processing" Jul 26 18:44:58.325759 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325692 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdc95c03-788c-4cfd-8f23-a507331bec41" containerName="payload-processing" Jul 26 18:44:58.325759 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325697 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdc95c03-788c-4cfd-8f23-a507331bec41" containerName="payload-processing" Jul 26 18:44:58.325759 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325709 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6594d7c-e383-4b97-bb10-2e5a61bb94d5" containerName="payload-pre-processing" Jul 26 18:44:58.325759 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325714 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6594d7c-e383-4b97-bb10-2e5a61bb94d5" containerName="payload-pre-processing" Jul 26 18:44:58.325759 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325720 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ec20e9f-b70c-4b2c-83bd-ad670ea41d72" containerName="payload-pre-processing" Jul 26 18:44:58.325759 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325725 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ec20e9f-b70c-4b2c-83bd-ad670ea41d72" containerName="payload-pre-processing" Jul 26 18:44:58.325759 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325732 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e500bed6-796b-408c-8a18-544c385e561e" containerName="payload-processing" Jul 26 18:44:58.325759 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325737 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e500bed6-796b-408c-8a18-544c385e561e" containerName="payload-processing" Jul 26 18:44:58.325759 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325745 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10eab379-287a-4515-94b4-bdaa580ba9f8" containerName="payload-processing" Jul 26 18:44:58.325759 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325750 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="10eab379-287a-4515-94b4-bdaa580ba9f8" containerName="payload-processing" Jul 26 18:44:58.325759 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325759 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="092c5c3d-c634-43c2-875b-5f7457bf3205" containerName="payload-pre-processing" Jul 26 18:44:58.325759 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325764 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="092c5c3d-c634-43c2-875b-5f7457bf3205" containerName="payload-pre-processing" Jul 26 18:44:58.326412 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325814 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a573d182-06bd-4ee1-bce8-17ef92641caf" containerName="payload-pre-processing" Jul 26 18:44:58.326412 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325821 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e500bed6-796b-408c-8a18-544c385e561e" containerName="payload-processing" Jul 26 18:44:58.326412 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325827 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6594d7c-e383-4b97-bb10-2e5a61bb94d5" containerName="payload-pre-processing" Jul 26 18:44:58.326412 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325833 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d589aff0-cdbd-43c1-ae2a-445bb9af45d6" containerName="payload-processing" Jul 26 18:44:58.326412 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325841 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ec20e9f-b70c-4b2c-83bd-ad670ea41d72" containerName="payload-pre-processing" Jul 26 18:44:58.326412 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325847 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdc95c03-788c-4cfd-8f23-a507331bec41" containerName="payload-processing" Jul 26 18:44:58.326412 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325852 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="10eab379-287a-4515-94b4-bdaa580ba9f8" containerName="payload-processing" Jul 26 18:44:58.326412 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325858 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="092c5c3d-c634-43c2-875b-5f7457bf3205" containerName="payload-pre-processing" Jul 26 18:44:58.326412 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325863 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="083d7d91-1568-4f92-a707-53ebd763fdb9" containerName="payload-processing" Jul 26 18:44:58.326412 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.325869 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4803572b-de48-4248-9c41-157759c3f2f4" containerName="payload-pre-processing" Jul 26 18:44:58.328892 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.328877 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" Jul 26 18:44:58.331164 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.331141 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wprln\"/\"default-dockercfg-6tfqw\"" Jul 26 18:44:58.331285 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.331141 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wprln\"/\"openshift-service-ca.crt\"" Jul 26 18:44:58.331949 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.331931 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wprln\"/\"kube-root-ca.crt\"" Jul 26 18:44:58.336122 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.335793 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5"] Jul 26 18:44:58.409839 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.409797 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/927eb52a-b064-4dbf-9ec3-70bcfce53ae0-sys\") pod \"perf-node-gather-daemonset-glzl5\" (UID: \"927eb52a-b064-4dbf-9ec3-70bcfce53ae0\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" Jul 26 18:44:58.410038 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.409866 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrx78\" (UniqueName: \"kubernetes.io/projected/927eb52a-b064-4dbf-9ec3-70bcfce53ae0-kube-api-access-nrx78\") pod \"perf-node-gather-daemonset-glzl5\" (UID: \"927eb52a-b064-4dbf-9ec3-70bcfce53ae0\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" Jul 26 18:44:58.410038 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.409909 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/927eb52a-b064-4dbf-9ec3-70bcfce53ae0-podres\") pod \"perf-node-gather-daemonset-glzl5\" (UID: \"927eb52a-b064-4dbf-9ec3-70bcfce53ae0\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" Jul 26 18:44:58.410038 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.409950 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/927eb52a-b064-4dbf-9ec3-70bcfce53ae0-proc\") pod \"perf-node-gather-daemonset-glzl5\" (UID: \"927eb52a-b064-4dbf-9ec3-70bcfce53ae0\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" Jul 26 18:44:58.410198 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.410071 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/927eb52a-b064-4dbf-9ec3-70bcfce53ae0-lib-modules\") pod \"perf-node-gather-daemonset-glzl5\" (UID: \"927eb52a-b064-4dbf-9ec3-70bcfce53ae0\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" Jul 26 18:44:58.511501 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.511462 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/927eb52a-b064-4dbf-9ec3-70bcfce53ae0-podres\") pod \"perf-node-gather-daemonset-glzl5\" (UID: \"927eb52a-b064-4dbf-9ec3-70bcfce53ae0\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" Jul 26 18:44:58.511710 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.511521 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/927eb52a-b064-4dbf-9ec3-70bcfce53ae0-proc\") pod \"perf-node-gather-daemonset-glzl5\" (UID: \"927eb52a-b064-4dbf-9ec3-70bcfce53ae0\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" Jul 26 18:44:58.511710 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.511562 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/927eb52a-b064-4dbf-9ec3-70bcfce53ae0-lib-modules\") pod \"perf-node-gather-daemonset-glzl5\" (UID: \"927eb52a-b064-4dbf-9ec3-70bcfce53ae0\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" Jul 26 18:44:58.511710 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.511596 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/927eb52a-b064-4dbf-9ec3-70bcfce53ae0-sys\") pod \"perf-node-gather-daemonset-glzl5\" (UID: \"927eb52a-b064-4dbf-9ec3-70bcfce53ae0\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" Jul 26 18:44:58.511710 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.511635 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrx78\" (UniqueName: \"kubernetes.io/projected/927eb52a-b064-4dbf-9ec3-70bcfce53ae0-kube-api-access-nrx78\") pod \"perf-node-gather-daemonset-glzl5\" (UID: \"927eb52a-b064-4dbf-9ec3-70bcfce53ae0\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" Jul 26 18:44:58.511710 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.511636 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/927eb52a-b064-4dbf-9ec3-70bcfce53ae0-podres\") pod \"perf-node-gather-daemonset-glzl5\" (UID: \"927eb52a-b064-4dbf-9ec3-70bcfce53ae0\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" Jul 26 18:44:58.511710 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.511658 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/927eb52a-b064-4dbf-9ec3-70bcfce53ae0-proc\") pod \"perf-node-gather-daemonset-glzl5\" (UID: \"927eb52a-b064-4dbf-9ec3-70bcfce53ae0\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" Jul 26 18:44:58.511910 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.511745 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/927eb52a-b064-4dbf-9ec3-70bcfce53ae0-sys\") pod \"perf-node-gather-daemonset-glzl5\" (UID: \"927eb52a-b064-4dbf-9ec3-70bcfce53ae0\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" Jul 26 18:44:58.511910 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.511749 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/927eb52a-b064-4dbf-9ec3-70bcfce53ae0-lib-modules\") pod \"perf-node-gather-daemonset-glzl5\" (UID: \"927eb52a-b064-4dbf-9ec3-70bcfce53ae0\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" Jul 26 18:44:58.519488 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.519466 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrx78\" (UniqueName: \"kubernetes.io/projected/927eb52a-b064-4dbf-9ec3-70bcfce53ae0-kube-api-access-nrx78\") pod \"perf-node-gather-daemonset-glzl5\" (UID: \"927eb52a-b064-4dbf-9ec3-70bcfce53ae0\") " pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" Jul 26 18:44:58.639945 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.639853 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" Jul 26 18:44:58.780112 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:58.774550 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5"] Jul 26 18:44:59.231313 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:59.231275 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" event={"ID":"927eb52a-b064-4dbf-9ec3-70bcfce53ae0","Type":"ContainerStarted","Data":"717b04250f0f76fbeeba6951ee75e038c5578f978a123353b9f502acbd26572a"} Jul 26 18:44:59.231477 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:59.231336 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" event={"ID":"927eb52a-b064-4dbf-9ec3-70bcfce53ae0","Type":"ContainerStarted","Data":"950aa43a2ca3ff61dfdca03f7936274fe13e3e1aa9601e30d1e118710af5dd1a"} Jul 26 18:44:59.231477 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:59.231380 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" Jul 26 18:44:59.247038 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:44:59.246987 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" podStartSLOduration=1.246971234 podStartE2EDuration="1.246971234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:44:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:44:59.245628979 +0000 UTC m=+3353.198284693" watchObservedRunningTime="2026-07-26 18:44:59.246971234 +0000 UTC m=+3353.199626922" Jul 26 18:45:00.137648 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:00.137616 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751525-s6hrs"] Jul 26 18:45:00.140900 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:00.140881 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-s6hrs" Jul 26 18:45:00.143495 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:00.143473 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-swzk5\"" Jul 26 18:45:00.147436 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:00.147404 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751525-s6hrs"] Jul 26 18:45:00.151923 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:00.151902 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-s6hrs" Jul 26 18:45:00.285138 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:00.285111 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751525-s6hrs"] Jul 26 18:45:00.287447 ip-10-0-128-209 kubenswrapper[2559]: W0726 18:45:00.287419 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71fb27d2_fbaa_4e6f_be8c_a37751400fd3.slice/crio-a593c0218aee4646ffdb0a3af1c2033970d3e7b6f39bfe29759c180ce2094c5b WatchSource:0}: Error finding container a593c0218aee4646ffdb0a3af1c2033970d3e7b6f39bfe29759c180ce2094c5b: Status 404 returned error can't find the container with id a593c0218aee4646ffdb0a3af1c2033970d3e7b6f39bfe29759c180ce2094c5b Jul 26 18:45:01.243612 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:01.243580 2559 generic.go:356] "Generic (PLEG): container finished" podID="71fb27d2-fbaa-4e6f-be8c-a37751400fd3" containerID="2790e15a39b9038177fc74585ab80facb78e664135ef299a851eb0f6970b1201" exitCode=0 Jul 26 18:45:01.244019 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:01.243660 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-s6hrs" event={"ID":"71fb27d2-fbaa-4e6f-be8c-a37751400fd3","Type":"ContainerDied","Data":"2790e15a39b9038177fc74585ab80facb78e664135ef299a851eb0f6970b1201"} Jul 26 18:45:01.244019 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:01.243715 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-s6hrs" event={"ID":"71fb27d2-fbaa-4e6f-be8c-a37751400fd3","Type":"ContainerStarted","Data":"a593c0218aee4646ffdb0a3af1c2033970d3e7b6f39bfe29759c180ce2094c5b"} Jul 26 18:45:01.334417 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:01.334389 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5q7q7_1f1f2371-6ad2-4065-929a-d4e87827fa52/dns/0.log" Jul 26 18:45:01.357841 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:01.357814 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5q7q7_1f1f2371-6ad2-4065-929a-d4e87827fa52/kube-rbac-proxy/0.log" Jul 26 18:45:01.435559 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:01.435488 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2575c_e0cf3a10-8a54-45ad-aafa-bb7175208585/dns-node-resolver/0.log" Jul 26 18:45:02.045971 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:02.045938 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7896856b5b-8tdxc_037b4a96-1fc7-46a7-beca-05ab6867c352/registry/0.log" Jul 26 18:45:02.076735 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:02.076704 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bh5vf_a5ef698f-5999-4902-9fe6-57b185586841/node-ca/0.log" Jul 26 18:45:02.392570 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:02.392546 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-s6hrs" Jul 26 18:45:03.254584 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:03.254529 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-s6hrs" Jul 26 18:45:03.254815 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:03.254523 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-s6hrs" event={"ID":"71fb27d2-fbaa-4e6f-be8c-a37751400fd3","Type":"ContainerDied","Data":"a593c0218aee4646ffdb0a3af1c2033970d3e7b6f39bfe29759c180ce2094c5b"} Jul 26 18:45:03.254815 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:03.254750 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a593c0218aee4646ffdb0a3af1c2033970d3e7b6f39bfe29759c180ce2094c5b" Jul 26 18:45:03.328268 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:03.328234 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-xmcxd_f3875b31-ffe3-48dd-b240-0e799813a0ef/discovery/0.log" Jul 26 18:45:03.349834 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:03.349809 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6855b89bd5-2586g_31bcded3-9797-486d-b175-9ef1fc6d0b1a/kube-auth-proxy/0.log" Jul 26 18:45:04.382616 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:04.382587 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bgs6z_c3ba442d-4651-44c3-b29d-14d536188970/serve-healthcheck-canary/0.log" Jul 26 18:45:05.000614 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:05.000580 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-tsbr9_f8c35763-fccd-40d0-8109-a5b681b9f675/insights-operator/0.log" Jul 26 18:45:05.095932 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:05.095900 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gfwst_67f55fe3-f055-4bd0-8728-bcb5fa86db98/kube-rbac-proxy/0.log" Jul 26 18:45:05.117007 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:05.116979 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gfwst_67f55fe3-f055-4bd0-8728-bcb5fa86db98/exporter/0.log" Jul 26 18:45:05.137633 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:05.137596 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gfwst_67f55fe3-f055-4bd0-8728-bcb5fa86db98/extractor/0.log" Jul 26 18:45:05.248104 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:05.248079 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wprln/perf-node-gather-daemonset-glzl5" Jul 26 18:45:08.072662 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:08.072580 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67b95f7b6f-krcq4_2098ad1a-2857-4700-ab98-922266c839f5/manager/0.log" Jul 26 18:45:08.093297 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:08.093271 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-jjvgl_63332e8b-dfa5-4aeb-bcb9-b7166a0c7a23/server/0.log" Jul 26 18:45:08.296648 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:08.296618 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b79b7df6c-rjcmd_2cec0f48-bf3f-4a86-b72d-602edc087625/manager/0.log" Jul 26 18:45:08.319756 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:08.319726 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b79b7df6c-rjcmd_2cec0f48-bf3f-4a86-b72d-602edc087625/copy-manifests/0.log" Jul 26 18:45:09.104216 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:09.104180 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-xmcxd_f3875b31-ffe3-48dd-b240-0e799813a0ef/discovery/0.log" Jul 26 18:45:09.116860 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:09.116834 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6855b89bd5-2586g_31bcded3-9797-486d-b175-9ef1fc6d0b1a/kube-auth-proxy/0.log" Jul 26 18:45:09.695506 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:09.695472 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-4ndxz_30b68744-d39f-40e3-8856-26894de4f8a3/openshift-lws-operator/0.log" Jul 26 18:45:10.448353 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:10.448307 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4sdbx_5c23228c-55bb-49e3-b76f-d22199c6a153/konnectivity-agent/0.log" limit=1024 Jul 26 18:45:10.921169 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:10.921131 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-xmcxd_f3875b31-ffe3-48dd-b240-0e799813a0ef/discovery/0.log" Jul 26 18:45:10.932715 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:10.932690 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6855b89bd5-2586g_31bcded3-9797-486d-b175-9ef1fc6d0b1a/kube-auth-proxy/0.log" Jul 26 18:45:14.418138 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:14.418063 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-vzm9r_650616e9-345b-43fd-aab6-895e5af929d5/migrator/0.log" Jul 26 18:45:14.435853 ip-10-0-128-209 kubenswrapper[2559]: I0726 18:45:14.435823 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-vzm9r_650616e9-345b-43fd-aab6-895e5af929d5/graceful-termination/0.log"