Jul 29 20:25:04.376313 ip-10-0-138-67 systemd[1]: Starting Kubernetes Kubelet... Jul 29 20:25:04.730779 ip-10-0-138-67 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 20:25:04.730779 ip-10-0-138-67 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 29 20:25:04.730779 ip-10-0-138-67 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 20:25:04.730779 ip-10-0-138-67 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 29 20:25:04.730779 ip-10-0-138-67 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 20:25:04.732692 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.732585 2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 29 20:25:04.736582 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736567 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 20:25:04.736582 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736582 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 20:25:04.736644 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736586 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 20:25:04.736644 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736589 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 20:25:04.736644 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736593 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 20:25:04.736644 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736595 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 20:25:04.736644 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736598 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 20:25:04.736644 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736601 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 20:25:04.736644 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736604 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 20:25:04.736644 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736607 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 20:25:04.736644 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736610 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 20:25:04.736644 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736613 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 20:25:04.736644 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736616 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 20:25:04.736644 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736618 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 20:25:04.736644 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736621 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 20:25:04.736644 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736624 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 20:25:04.736644 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736626 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 20:25:04.736644 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736629 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 20:25:04.736644 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736632 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 20:25:04.736644 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736635 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 20:25:04.736644 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736638 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 20:25:04.736644 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736640 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 20:25:04.737109 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736643 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 20:25:04.737109 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736645 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 20:25:04.737109 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736648 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 20:25:04.737109 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736651 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 20:25:04.737109 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736654 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 20:25:04.737109 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736657 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 20:25:04.737109 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736659 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 20:25:04.737109 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736662 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 20:25:04.737109 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736665 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 20:25:04.737109 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736667 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 20:25:04.737109 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736669 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 20:25:04.737109 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736672 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 20:25:04.737109 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736675 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 20:25:04.737109 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736677 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 20:25:04.737109 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736679 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 20:25:04.737109 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736682 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 20:25:04.737109 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736684 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 20:25:04.737109 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736686 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 20:25:04.737109 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736689 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 20:25:04.737109 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736692 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 20:25:04.737580 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736696 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 20:25:04.737580 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736700 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 20:25:04.737580 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736702 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 20:25:04.737580 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736705 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 20:25:04.737580 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736708 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 20:25:04.737580 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736711 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 20:25:04.737580 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736713 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 29 20:25:04.737580 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736716 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 20:25:04.737580 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736718 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 20:25:04.737580 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736722 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 20:25:04.737580 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736726 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 20:25:04.737580 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736729 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 20:25:04.737580 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736733 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 20:25:04.737580 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736736 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 20:25:04.737580 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736738 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 20:25:04.737580 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736741 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 20:25:04.737580 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736743 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 20:25:04.737580 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736746 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 20:25:04.737580 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736749 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 20:25:04.738021 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736752 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 20:25:04.738021 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736754 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 20:25:04.738021 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736758 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 20:25:04.738021 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736761 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 20:25:04.738021 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736764 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 20:25:04.738021 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736767 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 20:25:04.738021 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736770 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 20:25:04.738021 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736773 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 20:25:04.738021 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736775 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 20:25:04.738021 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736778 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 20:25:04.738021 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736790 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 20:25:04.738021 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736792 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 20:25:04.738021 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736795 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 20:25:04.738021 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736798 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 20:25:04.738021 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736800 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 20:25:04.738021 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736803 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 20:25:04.738021 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736805 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 20:25:04.738021 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736807 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 20:25:04.738021 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736810 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 20:25:04.738513 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736813 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 20:25:04.738513 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736815 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 20:25:04.738513 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736818 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 20:25:04.738513 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736820 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 20:25:04.738513 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736822 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 20:25:04.738513 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736826 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 20:25:04.738513 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736829 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 20:25:04.738513 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736831 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 20:25:04.738513 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736834 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 20:25:04.738513 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736837 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 20:25:04.738513 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736839 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 20:25:04.738513 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736842 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 20:25:04.738513 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736845 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 20:25:04.738513 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736847 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 20:25:04.738513 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736850 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 20:25:04.738513 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736852 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 20:25:04.738513 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736854 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 20:25:04.738513 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736857 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 20:25:04.738513 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.736859 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 20:25:04.738513 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738373 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 20:25:04.738982 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738380 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 20:25:04.738982 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738383 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 20:25:04.738982 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738386 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 20:25:04.738982 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738389 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 20:25:04.738982 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738393 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 20:25:04.738982 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738397 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 20:25:04.738982 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738401 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 29 20:25:04.738982 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738404 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 20:25:04.738982 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738407 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 20:25:04.738982 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738410 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 20:25:04.738982 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738413 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 20:25:04.738982 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738416 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 20:25:04.738982 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738418 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 20:25:04.738982 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738421 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 20:25:04.738982 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738424 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 20:25:04.738982 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738428 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 20:25:04.738982 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738431 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 20:25:04.738982 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738435 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 20:25:04.738982 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738438 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 20:25:04.739443 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738441 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 20:25:04.739443 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738443 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 20:25:04.739443 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738446 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 20:25:04.739443 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738448 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 20:25:04.739443 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738451 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 20:25:04.739443 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738454 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 20:25:04.739443 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738456 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 20:25:04.739443 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738459 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 20:25:04.739443 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738461 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 20:25:04.739443 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738464 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 20:25:04.739443 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738467 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 20:25:04.739443 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738469 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 20:25:04.739443 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738473 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 20:25:04.739443 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738476 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 20:25:04.739443 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738478 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 20:25:04.739443 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738481 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 20:25:04.739443 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738483 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 20:25:04.739443 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738486 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 20:25:04.739443 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738494 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 20:25:04.739885 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738497 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 20:25:04.739885 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738501 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 20:25:04.739885 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738504 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 20:25:04.739885 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738507 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 20:25:04.739885 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738510 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 20:25:04.739885 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738512 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 20:25:04.739885 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738515 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 20:25:04.739885 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738518 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 20:25:04.739885 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738521 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 20:25:04.739885 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738523 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 20:25:04.739885 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738526 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 20:25:04.739885 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738529 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 20:25:04.739885 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738531 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 20:25:04.739885 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738534 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 20:25:04.739885 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738537 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 20:25:04.739885 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738539 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 20:25:04.739885 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738542 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 20:25:04.739885 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738544 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 20:25:04.739885 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738547 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 20:25:04.739885 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738549 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 20:25:04.740360 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738552 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 20:25:04.740360 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738555 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 20:25:04.740360 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738557 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 20:25:04.740360 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738560 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 20:25:04.740360 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738563 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 20:25:04.740360 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738565 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 20:25:04.740360 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738568 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 20:25:04.740360 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738571 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 20:25:04.740360 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738573 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 20:25:04.740360 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738576 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 20:25:04.740360 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738578 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 20:25:04.740360 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738581 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 20:25:04.740360 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738584 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 20:25:04.740360 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738586 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 20:25:04.740360 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738588 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 20:25:04.740360 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738591 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 20:25:04.740360 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738593 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 20:25:04.740360 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738596 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 20:25:04.740360 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738598 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 20:25:04.740360 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738601 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 20:25:04.740827 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738605 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 20:25:04.740827 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738608 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 20:25:04.740827 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738610 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 20:25:04.740827 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738613 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 20:25:04.740827 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738615 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 20:25:04.740827 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738617 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 20:25:04.740827 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738620 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 20:25:04.740827 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738623 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 20:25:04.740827 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738625 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 20:25:04.740827 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738628 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 20:25:04.740827 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738631 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 20:25:04.740827 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738634 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 20:25:04.740827 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738636 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 20:25:04.740827 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738639 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 20:25:04.740827 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738641 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 20:25:04.740827 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738644 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 20:25:04.740827 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738646 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 20:25:04.740827 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738648 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 20:25:04.740827 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738651 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 20:25:04.741276 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738653 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 20:25:04.741276 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738987 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 20:25:04.741276 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738992 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 20:25:04.741276 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738995 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 20:25:04.741276 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.738997 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 20:25:04.741276 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739000 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 20:25:04.741276 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739003 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 20:25:04.741276 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739005 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 20:25:04.741276 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739008 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 20:25:04.741276 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739010 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 20:25:04.741276 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739013 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 20:25:04.741276 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739015 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 20:25:04.741276 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739018 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 20:25:04.741276 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739022 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 20:25:04.741276 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739025 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 20:25:04.741276 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739027 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 20:25:04.741276 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739030 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 20:25:04.741276 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739032 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 20:25:04.741276 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739035 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 20:25:04.741276 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739037 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 20:25:04.741276 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739040 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 20:25:04.741779 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739063 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 20:25:04.741779 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739067 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 20:25:04.741779 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739070 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 20:25:04.741779 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739073 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 20:25:04.741779 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739076 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 20:25:04.741779 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739079 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 20:25:04.741779 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739081 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 20:25:04.741779 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739084 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 20:25:04.741779 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739086 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 20:25:04.741779 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739089 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 20:25:04.741779 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739092 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 29 20:25:04.741779 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739094 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 20:25:04.741779 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739096 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 20:25:04.741779 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739099 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 20:25:04.741779 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739102 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 20:25:04.741779 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739105 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 20:25:04.741779 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739108 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 20:25:04.741779 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739111 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 20:25:04.741779 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739113 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 20:25:04.742237 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739115 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 20:25:04.742237 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739118 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 20:25:04.742237 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739121 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 20:25:04.742237 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739123 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 20:25:04.742237 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739125 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 20:25:04.742237 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739130 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 20:25:04.742237 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739132 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 20:25:04.742237 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739135 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 20:25:04.742237 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739137 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 20:25:04.742237 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739140 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 20:25:04.742237 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739142 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 20:25:04.742237 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739145 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 20:25:04.742237 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739147 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 20:25:04.742237 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739150 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 20:25:04.742237 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739152 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 20:25:04.742237 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739155 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 20:25:04.742237 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739158 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 20:25:04.742237 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739162 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 20:25:04.742237 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739165 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 20:25:04.742707 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739168 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 20:25:04.742707 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739171 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 20:25:04.742707 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739174 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 20:25:04.742707 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739176 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 20:25:04.742707 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739179 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 20:25:04.742707 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739182 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 20:25:04.742707 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739185 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 20:25:04.742707 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739188 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 20:25:04.742707 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739191 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 20:25:04.742707 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739193 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 20:25:04.742707 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739196 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 20:25:04.742707 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739198 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 20:25:04.742707 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739201 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 20:25:04.742707 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739205 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 20:25:04.742707 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739207 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 20:25:04.742707 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739210 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 20:25:04.742707 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739212 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 20:25:04.742707 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739214 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 20:25:04.742707 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739218 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 20:25:04.742707 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739221 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 20:25:04.743188 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739223 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 20:25:04.743188 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739226 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 20:25:04.743188 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739228 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 20:25:04.743188 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739231 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 20:25:04.743188 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739233 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 20:25:04.743188 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739236 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 20:25:04.743188 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739238 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 20:25:04.743188 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739241 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 20:25:04.743188 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739244 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 20:25:04.743188 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739246 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 20:25:04.743188 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739249 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 20:25:04.743188 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739251 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 20:25:04.743188 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739253 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 20:25:04.743188 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739256 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 20:25:04.743188 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739258 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 20:25:04.743188 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739261 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 20:25:04.743188 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739263 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 20:25:04.743188 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739265 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 20:25:04.743188 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739268 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 20:25:04.743188 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739270 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739273 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739341 2559 flags.go:64] FLAG: --address="0.0.0.0" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739347 2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739355 2559 flags.go:64] FLAG: --anonymous-auth="true" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739361 2559 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739369 2559 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739373 2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739384 2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739389 2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739392 2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739396 2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739400 2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739403 2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739406 2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739409 2559 flags.go:64] FLAG: --cgroup-root="" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739412 2559 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739414 2559 flags.go:64] FLAG: --client-ca-file="" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739417 2559 flags.go:64] FLAG: --cloud-provider="external" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739421 2559 flags.go:64] FLAG: --cluster-dns="[]" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739426 2559 flags.go:64] FLAG: --cluster-domain="" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739429 2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739432 2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739435 2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 29 20:25:04.743659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739438 2559 flags.go:64] FLAG: --container-log-max-files="5" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739445 2559 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739448 2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739452 2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739455 2559 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739458 2559 flags.go:64] FLAG: --contention-profiling="false" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739461 2559 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739464 2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739467 2559 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739469 2559 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739473 2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739476 2559 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739479 2559 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739482 2559 flags.go:64] FLAG: --enable-load-reader="false" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739485 2559 flags.go:64] FLAG: --enable-server="true" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739488 2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739492 2559 flags.go:64] FLAG: --event-burst="100" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739495 2559 flags.go:64] FLAG: --event-qps="50" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739498 2559 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739502 2559 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739505 2559 flags.go:64] FLAG: --eviction-hard="" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739508 2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739511 2559 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739514 2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739517 2559 flags.go:64] FLAG: --eviction-soft="" Jul 29 20:25:04.744341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739520 2559 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739522 2559 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739527 2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739530 2559 flags.go:64] FLAG: --experimental-mounter-path="" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739533 2559 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739535 2559 flags.go:64] FLAG: --fail-swap-on="true" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739538 2559 flags.go:64] FLAG: --feature-gates="" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739541 2559 flags.go:64] FLAG: --file-check-frequency="20s" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739544 2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739547 2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739550 2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739553 2559 flags.go:64] FLAG: --healthz-port="10248" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739556 2559 flags.go:64] FLAG: --help="false" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739559 2559 flags.go:64] FLAG: --hostname-override="ip-10-0-138-67.ec2.internal" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739562 2559 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739564 2559 flags.go:64] FLAG: --http-check-frequency="20s" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739567 2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739570 2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739573 2559 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739576 2559 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739579 2559 flags.go:64] FLAG: --image-service-endpoint="" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739581 2559 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739584 2559 flags.go:64] FLAG: --kube-api-burst="100" Jul 29 20:25:04.744943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739587 2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739590 2559 flags.go:64] FLAG: --kube-api-qps="50" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739593 2559 flags.go:64] FLAG: --kube-reserved="" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739597 2559 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739599 2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739602 2559 flags.go:64] FLAG: --kubelet-cgroups="" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739605 2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739608 2559 flags.go:64] FLAG: --lock-file="" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739610 2559 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739613 2559 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739616 2559 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739622 2559 flags.go:64] FLAG: --log-json-split-stream="false" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739625 2559 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739627 2559 flags.go:64] FLAG: --log-text-split-stream="false" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739630 2559 flags.go:64] FLAG: --logging-format="text" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739633 2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739636 2559 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739639 2559 flags.go:64] FLAG: --manifest-url="" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739642 2559 flags.go:64] FLAG: --manifest-url-header="" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739646 2559 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739648 2559 flags.go:64] FLAG: --max-open-files="1000000" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739652 2559 flags.go:64] FLAG: --max-pods="110" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739655 2559 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739658 2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739660 2559 flags.go:64] FLAG: --memory-manager-policy="None" Jul 29 20:25:04.745506 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739663 2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739666 2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739669 2559 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739672 2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739678 2559 flags.go:64] FLAG: --node-status-max-images="50" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739681 2559 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739684 2559 flags.go:64] FLAG: --oom-score-adj="-999" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739687 2559 flags.go:64] FLAG: --pod-cidr="" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739690 2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739695 2559 flags.go:64] FLAG: --pod-manifest-path="" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739698 2559 flags.go:64] FLAG: --pod-max-pids="-1" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739702 2559 flags.go:64] FLAG: --pods-per-core="0" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739705 2559 flags.go:64] FLAG: --port="10250" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739708 2559 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739711 2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-063118c44a8fe1af5" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739714 2559 flags.go:64] FLAG: --qos-reserved="" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739716 2559 flags.go:64] FLAG: --read-only-port="10255" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739719 2559 flags.go:64] FLAG: --register-node="true" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739723 2559 flags.go:64] FLAG: --register-with-taints="" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739727 2559 flags.go:64] FLAG: --registry-burst="10" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739730 2559 flags.go:64] FLAG: --registry-qps="5" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739732 2559 flags.go:64] FLAG: --reserved-cpus="" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739735 2559 flags.go:64] FLAG: --reserved-memory="" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739738 2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739741 2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 29 20:25:04.746144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739744 2559 flags.go:64] FLAG: --rotate-certificates="false" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739747 2559 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739749 2559 flags.go:64] FLAG: --runonce="false" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739752 2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739755 2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739758 2559 flags.go:64] FLAG: --seccomp-default="false" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739760 2559 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739763 2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739766 2559 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739769 2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739772 2559 flags.go:64] FLAG: --storage-driver-password="root" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739774 2559 flags.go:64] FLAG: --storage-driver-secure="false" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739777 2559 flags.go:64] FLAG: --storage-driver-table="stats" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739779 2559 flags.go:64] FLAG: --storage-driver-user="root" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739782 2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739785 2559 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739788 2559 flags.go:64] FLAG: --system-cgroups="" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739791 2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739798 2559 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739800 2559 flags.go:64] FLAG: --tls-cert-file="" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739803 2559 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739807 2559 flags.go:64] FLAG: --tls-min-version="" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739809 2559 flags.go:64] FLAG: --tls-private-key-file="" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739812 2559 flags.go:64] FLAG: --topology-manager-policy="none" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739815 2559 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 29 20:25:04.746731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739819 2559 flags.go:64] FLAG: --topology-manager-scope="container" Jul 29 20:25:04.747334 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739822 2559 flags.go:64] FLAG: --v="2" Jul 29 20:25:04.747334 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739826 2559 flags.go:64] FLAG: --version="false" Jul 29 20:25:04.747334 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739830 2559 flags.go:64] FLAG: --vmodule="" Jul 29 20:25:04.747334 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739833 2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 29 20:25:04.747334 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.739836 2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 29 20:25:04.747334 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739947 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 20:25:04.747334 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739952 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 20:25:04.747334 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739956 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 20:25:04.747334 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739959 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 20:25:04.747334 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739962 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 20:25:04.747334 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739965 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 29 20:25:04.747334 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739968 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 20:25:04.747334 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739971 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 20:25:04.747334 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739973 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 20:25:04.747334 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739976 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 20:25:04.747334 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739980 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 20:25:04.747334 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739983 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 20:25:04.747334 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739986 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 20:25:04.747334 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739989 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 20:25:04.747334 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739992 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 20:25:04.747792 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739995 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 20:25:04.747792 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.739998 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 20:25:04.747792 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740001 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 20:25:04.747792 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740005 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 20:25:04.747792 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740008 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 20:25:04.747792 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740011 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 20:25:04.747792 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740013 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 20:25:04.747792 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740016 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 20:25:04.747792 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740018 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 20:25:04.747792 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740021 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 20:25:04.747792 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740023 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 20:25:04.747792 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740027 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 20:25:04.747792 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740029 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 20:25:04.747792 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740031 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 20:25:04.747792 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740034 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 20:25:04.747792 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740037 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 20:25:04.747792 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740039 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 20:25:04.747792 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740041 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 20:25:04.747792 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740056 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 20:25:04.747792 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740059 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 20:25:04.748286 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740061 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 20:25:04.748286 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740064 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 20:25:04.748286 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740066 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 20:25:04.748286 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740069 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 20:25:04.748286 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740071 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 20:25:04.748286 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740074 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 20:25:04.748286 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740076 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 20:25:04.748286 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740078 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 20:25:04.748286 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740081 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 20:25:04.748286 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740083 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 20:25:04.748286 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740086 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 20:25:04.748286 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740088 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 20:25:04.748286 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740091 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 20:25:04.748286 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740093 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 20:25:04.748286 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740095 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 20:25:04.748286 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740098 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 20:25:04.748286 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740101 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 20:25:04.748286 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740103 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 20:25:04.748286 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740106 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 20:25:04.748286 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740108 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 20:25:04.748747 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740111 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 20:25:04.748747 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740113 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 20:25:04.748747 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740116 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 20:25:04.748747 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740120 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 20:25:04.748747 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740122 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 20:25:04.748747 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740124 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 20:25:04.748747 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740127 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 20:25:04.748747 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740129 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 20:25:04.748747 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740131 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 20:25:04.748747 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740134 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 20:25:04.748747 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740136 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 20:25:04.748747 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740139 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 20:25:04.748747 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740141 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 20:25:04.748747 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740144 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 20:25:04.748747 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740146 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 20:25:04.748747 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740148 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 20:25:04.748747 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740151 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 20:25:04.748747 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740153 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 20:25:04.748747 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740155 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 20:25:04.748747 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740158 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 20:25:04.749215 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740160 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 20:25:04.749215 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740163 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 20:25:04.749215 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740165 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 20:25:04.749215 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740167 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 20:25:04.749215 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740170 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 20:25:04.749215 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740172 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 20:25:04.749215 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740175 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 20:25:04.749215 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740177 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 20:25:04.749215 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740179 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 20:25:04.749215 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740182 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 20:25:04.749215 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740184 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 20:25:04.749215 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740187 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 20:25:04.749215 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740189 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 20:25:04.749215 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740192 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 20:25:04.749215 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740194 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 20:25:04.749215 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740198 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 20:25:04.749215 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740200 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 20:25:04.749215 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740203 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 20:25:04.749215 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740205 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 20:25:04.749215 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740207 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 20:25:04.749691 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740210 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 20:25:04.749691 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740212 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 20:25:04.749691 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740215 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 20:25:04.749691 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.740217 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 20:25:04.749691 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.740225 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 20:25:04.749691 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.746570 2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 29 20:25:04.749691 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.746585 2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 29 20:25:04.749691 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.747324 2559 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 29 20:25:04.749691 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.747332 2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 29 20:25:04.749691 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747388 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 20:25:04.749691 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747394 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 20:25:04.749691 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747398 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 20:25:04.749691 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747401 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 20:25:04.749691 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747404 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 20:25:04.749691 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747407 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 20:25:04.750060 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747410 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 20:25:04.750060 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747413 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 20:25:04.750060 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747415 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 20:25:04.750060 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747418 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 20:25:04.750060 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747420 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 20:25:04.750060 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747423 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 20:25:04.750060 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747426 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 20:25:04.750060 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747429 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 20:25:04.750060 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747432 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 20:25:04.750060 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747435 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 20:25:04.750060 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747437 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 20:25:04.750060 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747440 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 20:25:04.750060 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747443 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 20:25:04.750060 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747445 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 20:25:04.750060 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747448 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 20:25:04.750060 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747452 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 20:25:04.750060 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747455 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 20:25:04.750060 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747458 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 20:25:04.750060 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747461 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 20:25:04.750500 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747463 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 20:25:04.750500 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747466 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 20:25:04.750500 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747468 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 20:25:04.750500 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747471 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 20:25:04.750500 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747474 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 20:25:04.750500 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747477 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 20:25:04.750500 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747480 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 20:25:04.750500 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747483 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 29 20:25:04.750500 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747485 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 20:25:04.750500 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747488 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 20:25:04.750500 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747491 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 20:25:04.750500 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747493 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 20:25:04.750500 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747496 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 20:25:04.750500 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747498 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 20:25:04.750500 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747501 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 20:25:04.750500 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747503 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 20:25:04.750500 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747506 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 20:25:04.750500 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747509 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 20:25:04.750500 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747511 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 20:25:04.750500 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747514 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 20:25:04.750958 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747519 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 20:25:04.750958 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747522 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 20:25:04.750958 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747525 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 20:25:04.750958 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747528 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 20:25:04.750958 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747531 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 20:25:04.750958 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747534 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 20:25:04.750958 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747536 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 20:25:04.750958 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747539 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 20:25:04.750958 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747542 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 20:25:04.750958 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747544 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 20:25:04.750958 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747547 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 20:25:04.750958 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747549 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 20:25:04.750958 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747552 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 20:25:04.750958 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747554 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 20:25:04.750958 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747557 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 20:25:04.750958 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747559 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 20:25:04.750958 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747563 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 20:25:04.750958 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747565 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 20:25:04.750958 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747568 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 20:25:04.751437 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747571 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 20:25:04.751437 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747573 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 20:25:04.751437 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747576 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 20:25:04.751437 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747578 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 20:25:04.751437 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747581 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 20:25:04.751437 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747583 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 20:25:04.751437 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747586 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 20:25:04.751437 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747588 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 20:25:04.751437 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747591 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 20:25:04.751437 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747593 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 20:25:04.751437 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747595 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 20:25:04.751437 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747598 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 20:25:04.751437 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747600 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 20:25:04.751437 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747603 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 20:25:04.751437 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747606 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 20:25:04.751437 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747608 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 20:25:04.751437 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747611 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 20:25:04.751437 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747613 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 20:25:04.751437 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747616 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 20:25:04.751437 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747618 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 20:25:04.751901 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747620 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 20:25:04.751901 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747623 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 20:25:04.751901 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747625 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 20:25:04.751901 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747628 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 20:25:04.751901 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747630 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 20:25:04.751901 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747632 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 20:25:04.751901 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747635 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 20:25:04.751901 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747638 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 20:25:04.751901 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747640 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 20:25:04.751901 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747644 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 20:25:04.751901 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747648 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 20:25:04.751901 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747651 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 20:25:04.751901 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747654 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 20:25:04.751901 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747657 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 20:25:04.751901 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747659 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 20:25:04.752385 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.747664 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 20:25:04.752385 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747769 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 20:25:04.752385 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747773 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 20:25:04.752385 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747776 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 20:25:04.752385 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747779 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 20:25:04.752385 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747781 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 20:25:04.752385 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747784 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 20:25:04.752385 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747787 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 20:25:04.752385 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747789 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 20:25:04.752385 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747793 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 20:25:04.752385 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747795 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 20:25:04.752385 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747798 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 20:25:04.752385 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747801 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 20:25:04.752385 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747804 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 20:25:04.752385 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747807 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 20:25:04.752729 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747809 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 20:25:04.752729 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747812 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 20:25:04.752729 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747814 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 20:25:04.752729 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747817 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 20:25:04.752729 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747819 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 20:25:04.752729 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747821 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 20:25:04.752729 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747824 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 20:25:04.752729 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747827 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 20:25:04.752729 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747829 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 20:25:04.752729 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747831 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 20:25:04.752729 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747834 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 20:25:04.752729 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747837 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 20:25:04.752729 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747839 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 20:25:04.752729 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747842 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 29 20:25:04.752729 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747844 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 20:25:04.752729 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747847 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 20:25:04.752729 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747849 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 20:25:04.752729 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747852 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 20:25:04.752729 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747854 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 20:25:04.753308 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747858 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 20:25:04.753308 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747861 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 20:25:04.753308 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747864 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 20:25:04.753308 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747867 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 20:25:04.753308 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747869 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 20:25:04.753308 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747872 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 20:25:04.753308 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747875 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 20:25:04.753308 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747877 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 20:25:04.753308 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747880 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 20:25:04.753308 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747882 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 20:25:04.753308 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747885 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 20:25:04.753308 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747889 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 20:25:04.753308 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747892 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 20:25:04.753308 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747894 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 20:25:04.753308 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747897 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 20:25:04.753308 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747900 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 20:25:04.753308 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747902 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 20:25:04.753308 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747905 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 20:25:04.753308 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747907 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 20:25:04.753842 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747910 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 20:25:04.753842 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747913 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 20:25:04.753842 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747915 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 20:25:04.753842 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747918 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 20:25:04.753842 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747920 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 20:25:04.753842 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747923 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 20:25:04.753842 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747926 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 20:25:04.753842 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747928 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 20:25:04.753842 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747931 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 20:25:04.753842 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747933 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 20:25:04.753842 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747936 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 20:25:04.753842 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747939 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 20:25:04.753842 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747941 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 20:25:04.753842 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747943 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 20:25:04.753842 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747946 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 20:25:04.753842 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747948 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 20:25:04.753842 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747951 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 20:25:04.753842 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747953 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 20:25:04.753842 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747955 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 20:25:04.753842 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747958 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 20:25:04.754458 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747960 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 20:25:04.754458 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747963 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 20:25:04.754458 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747965 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 20:25:04.754458 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747967 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 20:25:04.754458 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747970 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 20:25:04.754458 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747973 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 20:25:04.754458 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747975 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 20:25:04.754458 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747978 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 20:25:04.754458 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747980 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 20:25:04.754458 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747983 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 20:25:04.754458 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747985 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 20:25:04.754458 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747988 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 20:25:04.754458 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747991 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 20:25:04.754458 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747993 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 20:25:04.754458 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747996 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 20:25:04.754458 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.747998 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 20:25:04.754458 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.748000 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 20:25:04.754458 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.748003 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 20:25:04.754458 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.748005 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 20:25:04.754458 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.748008 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 20:25:04.755228 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.748010 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 20:25:04.755228 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.748013 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 20:25:04.755228 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.748016 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 20:25:04.755228 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.748019 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 20:25:04.755228 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.748021 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 20:25:04.755228 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.748024 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 20:25:04.755228 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:04.748026 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 20:25:04.755228 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.748031 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 20:25:04.755228 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.748144 2559 server.go:962] "Client rotation is on, will bootstrap in background" Jul 29 20:25:04.755228 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.749875 2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 29 20:25:04.755228 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.751013 2559 server.go:1019] "Starting client certificate rotation" Jul 29 20:25:04.755228 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.751080 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 29 20:25:04.755228 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.751145 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 29 20:25:04.755638 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.753097 2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 29 20:25:04.758870 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.758852 2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 29 20:25:04.768027 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.768006 2559 log.go:25] "Validated CRI v1 runtime API" Jul 29 20:25:04.769902 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.769890 2559 log.go:25] "Validated CRI v1 image API" Jul 29 20:25:04.771342 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.771326 2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 29 20:25:04.773441 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.773420 2559 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 d7f6fff4-bb2d-4d59-a62f-ef2e561dab19:/dev/nvme0n1p3 f9dbae9f-1fd5-451b-8a63-40c46f6a5954:/dev/nvme0n1p4] Jul 29 20:25:04.773521 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.773440 2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 29 20:25:04.778299 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.778184 2559 manager.go:216] Machine: {Timestamp:2026-07-29 20:25:04.776953846 +0000 UTC m=+0.292260060 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3205196 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2eba362f73294c8a842d9f71616a6e SystemUUID:ec2eba36-2f73-294c-8a84-2d9f71616a6e BootID:0736beaa-7005-4959-82b3-c1310af8375a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:13:28:a0:a5:f7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:13:28:a0:a5:f7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:0f:41:5b:e9:52 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 29 20:25:04.778299 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.778291 2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 29 20:25:04.778433 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.778423 2559 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 29 20:25:04.780326 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.780300 2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 29 20:25:04.780469 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.780328 2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-67.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 29 20:25:04.781094 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.781082 2559 topology_manager.go:138] "Creating topology manager with none policy" Jul 29 20:25:04.781155 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.781099 2559 container_manager_linux.go:309] "Creating device plugin manager" Jul 29 20:25:04.781155 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.781109 2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 29 20:25:04.781259 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.781244 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 29 20:25:04.782631 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.782618 2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 29 20:25:04.783260 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.783249 2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 29 20:25:04.784744 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.784731 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 29 20:25:04.784866 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.784855 2559 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 29 20:25:04.784926 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.784891 2559 kubelet.go:486] "Attempting to sync node with API server" Jul 29 20:25:04.784926 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.784905 2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 29 20:25:04.784926 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.784920 2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 29 20:25:04.785043 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.784932 2559 kubelet.go:398] "Adding apiserver pod source" Jul 29 20:25:04.785043 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.784945 2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 29 20:25:04.785315 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.785302 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 29 20:25:04.785380 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.785324 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 29 20:25:04.787233 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.787214 2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 29 20:25:04.788805 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.788791 2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 29 20:25:04.788879 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.788826 2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 29 20:25:04.790256 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.790237 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 29 20:25:04.790256 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.790257 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 29 20:25:04.790353 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.790263 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 29 20:25:04.790353 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.790269 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 29 20:25:04.790353 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.790277 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 29 20:25:04.790353 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.790282 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 29 20:25:04.790353 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.790291 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 29 20:25:04.790353 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.790296 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 29 20:25:04.790353 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.790302 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 29 20:25:04.790353 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.790307 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 29 20:25:04.790353 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.790319 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 29 20:25:04.790353 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.790324 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 29 20:25:04.791014 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.791006 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 29 20:25:04.791014 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.791014 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 29 20:25:04.794085 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.794069 2559 server.go:1268] "Started kubelet" Jul 29 20:25:04.794168 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.794135 2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 29 20:25:04.797892 ip-10-0-138-67 systemd[1]: Started Kubernetes Kubelet. Jul 29 20:25:04.798112 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.797972 2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 29 20:25:04.798112 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.798066 2559 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 29 20:25:04.799818 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.799803 2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 29 20:25:04.800305 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.800290 2559 server.go:310] "Adding debug handlers to kubelet server" Jul 29 20:25:04.800801 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:04.800777 2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 29 20:25:04.801754 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.801733 2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-67.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 29 20:25:04.801754 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:04.801734 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-67.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 29 20:25:04.801905 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:04.801767 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 29 20:25:04.804695 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:04.803850 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-67.ec2.internal.18c6dc76ba9c1350 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-67.ec2.internal,UID:ip-10-0-138-67.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-67.ec2.internal,},FirstTimestamp:2026-07-29 20:25:04.794022736 +0000 UTC m=+0.309328998,LastTimestamp:2026-07-29 20:25:04.794022736 +0000 UTC m=+0.309328998,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-67.ec2.internal,}" Jul 29 20:25:04.806668 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.806655 2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 29 20:25:04.806718 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.806674 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 29 20:25:04.806784 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:04.806769 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-67.ec2.internal\" not found" Jul 29 20:25:04.807224 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.807211 2559 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 29 20:25:04.807322 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.807228 2559 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 29 20:25:04.807322 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.807217 2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 29 20:25:04.807423 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.807362 2559 reconstruct.go:97] "Volume reconstruction finished" Jul 29 20:25:04.807423 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.807370 2559 reconciler.go:29] "Reconciler: start to sync state" Jul 29 20:25:04.807423 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.807418 2559 factory.go:55] Registering systemd factory Jul 29 20:25:04.807539 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.807434 2559 factory.go:223] Registration of the systemd container factory successfully Jul 29 20:25:04.807614 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.807598 2559 factory.go:153] Registering CRI-O factory Jul 29 20:25:04.807614 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.807613 2559 factory.go:223] Registration of the crio container factory successfully Jul 29 20:25:04.807762 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.807655 2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 29 20:25:04.807762 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.807675 2559 factory.go:103] Registering Raw factory Jul 29 20:25:04.807762 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.807689 2559 manager.go:1196] Started watching for new ooms in manager Jul 29 20:25:04.808160 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.808148 2559 manager.go:349] Starting recovery of all containers Jul 29 20:25:04.813090 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:04.813043 2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-67.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 29 20:25:04.813163 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:04.813123 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 29 20:25:04.818877 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.818850 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-68kb4" Jul 29 20:25:04.819573 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.819537 2559 manager.go:354] Recovery completed Jul 29 20:25:04.821000 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:04.820972 2559 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 29 20:25:04.823923 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.823910 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 20:25:04.824678 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.824662 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-68kb4" Jul 29 20:25:04.826235 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.826105 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeHasSufficientMemory" Jul 29 20:25:04.826235 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.826141 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 20:25:04.826235 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.826155 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeHasSufficientPID" Jul 29 20:25:04.826679 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.826664 2559 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 29 20:25:04.826679 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.826677 2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 29 20:25:04.826783 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.826689 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 20:25:04.826783 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.826693 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 29 20:25:04.827382 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.827370 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeHasSufficientMemory" Jul 29 20:25:04.827442 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.827389 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 20:25:04.827442 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.827400 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeHasSufficientPID" Jul 29 20:25:04.827686 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:04.827616 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-67.ec2.internal.18c6dc76bc85e6dd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-67.ec2.internal,UID:ip-10-0-138-67.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-67.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-67.ec2.internal,},FirstTimestamp:2026-07-29 20:25:04.826123997 +0000 UTC m=+0.341430216,LastTimestamp:2026-07-29 20:25:04.826123997 +0000 UTC m=+0.341430216,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-67.ec2.internal,}" Jul 29 20:25:04.828351 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.828339 2559 policy_none.go:49] "None policy: Start" Jul 29 20:25:04.828411 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.828355 2559 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 29 20:25:04.828411 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.828365 2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 29 20:25:04.829208 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.829198 2559 policy_none.go:47] "Start" Jul 29 20:25:04.869748 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.869580 2559 manager.go:337] "Starting Device Plugin manager" Jul 29 20:25:04.869857 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:04.869773 2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 29 20:25:04.869857 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.869786 2559 server.go:87] "Starting device plugin registration server" Jul 29 20:25:04.869997 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.869984 2559 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 29 20:25:04.870040 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.869999 2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 29 20:25:04.870116 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.870090 2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 29 20:25:04.870202 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.870189 2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 29 20:25:04.870202 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.870202 2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 29 20:25:04.870672 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:04.870651 2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 29 20:25:04.870749 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:04.870697 2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-67.ec2.internal\" not found" Jul 29 20:25:04.970271 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.970250 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 20:25:04.971160 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.971144 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeHasSufficientMemory" Jul 29 20:25:04.971229 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.971173 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 20:25:04.971229 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.971184 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeHasSufficientPID" Jul 29 20:25:04.971229 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.971213 2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-67.ec2.internal" Jul 29 20:25:04.979526 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:04.979509 2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-67.ec2.internal" Jul 29 20:25:04.979624 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:04.979534 2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-67.ec2.internal\": node \"ip-10-0-138-67.ec2.internal\" not found" Jul 29 20:25:04.998038 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:04.997987 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-67.ec2.internal\" not found" Jul 29 20:25:05.017796 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.017775 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 29 20:25:05.018851 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.018839 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 29 20:25:05.018901 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.018875 2559 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 29 20:25:05.018901 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.018894 2559 kubelet.go:2448] "Starting kubelet main sync loop" Jul 29 20:25:05.018975 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:05.018926 2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 29 20:25:05.022324 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.022305 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 20:25:05.098309 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:05.098286 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-67.ec2.internal\" not found" Jul 29 20:25:05.119083 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.119043 2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-67.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-67.ec2.internal"] Jul 29 20:25:05.119160 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.119121 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 20:25:05.120175 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.120158 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeHasSufficientMemory" Jul 29 20:25:05.120254 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.120189 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 20:25:05.120254 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.120202 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeHasSufficientPID" Jul 29 20:25:05.121723 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.121709 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 20:25:05.121853 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.121832 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-67.ec2.internal" Jul 29 20:25:05.121909 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.121869 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 20:25:05.122440 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.122423 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeHasSufficientMemory" Jul 29 20:25:05.122508 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.122450 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 20:25:05.122508 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.122462 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeHasSufficientPID" Jul 29 20:25:05.122508 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.122488 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeHasSufficientMemory" Jul 29 20:25:05.122508 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.122508 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 20:25:05.122644 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.122522 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeHasSufficientPID" Jul 29 20:25:05.123749 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.123733 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-67.ec2.internal" Jul 29 20:25:05.123838 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.123757 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 20:25:05.124462 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.124447 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeHasSufficientMemory" Jul 29 20:25:05.124560 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.124471 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 20:25:05.124560 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.124480 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeHasSufficientPID" Jul 29 20:25:05.148705 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:05.148682 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-67.ec2.internal\" not found" node="ip-10-0-138-67.ec2.internal" Jul 29 20:25:05.153141 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:05.153105 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-67.ec2.internal\" not found" node="ip-10-0-138-67.ec2.internal" Jul 29 20:25:05.198711 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:05.198690 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-67.ec2.internal\" not found" Jul 29 20:25:05.209390 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.209372 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eebf028fb5bb404136025b827cf76e9e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-67.ec2.internal\" (UID: \"eebf028fb5bb404136025b827cf76e9e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-67.ec2.internal" Jul 29 20:25:05.209445 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.209397 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/644f10f3f902f0c1cbb90a43f1a164ef-config\") pod \"kube-apiserver-proxy-ip-10-0-138-67.ec2.internal\" (UID: \"644f10f3f902f0c1cbb90a43f1a164ef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-67.ec2.internal" Jul 29 20:25:05.209445 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.209415 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eebf028fb5bb404136025b827cf76e9e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-67.ec2.internal\" (UID: \"eebf028fb5bb404136025b827cf76e9e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-67.ec2.internal" Jul 29 20:25:05.299182 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:05.299103 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-67.ec2.internal\" not found" Jul 29 20:25:05.310270 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.310252 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eebf028fb5bb404136025b827cf76e9e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-67.ec2.internal\" (UID: \"eebf028fb5bb404136025b827cf76e9e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-67.ec2.internal" Jul 29 20:25:05.310334 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.310322 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eebf028fb5bb404136025b827cf76e9e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-67.ec2.internal\" (UID: \"eebf028fb5bb404136025b827cf76e9e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-67.ec2.internal" Jul 29 20:25:05.310380 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.310340 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eebf028fb5bb404136025b827cf76e9e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-67.ec2.internal\" (UID: \"eebf028fb5bb404136025b827cf76e9e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-67.ec2.internal" Jul 29 20:25:05.310414 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.310381 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eebf028fb5bb404136025b827cf76e9e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-67.ec2.internal\" (UID: \"eebf028fb5bb404136025b827cf76e9e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-67.ec2.internal" Jul 29 20:25:05.310414 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.310392 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/644f10f3f902f0c1cbb90a43f1a164ef-config\") pod \"kube-apiserver-proxy-ip-10-0-138-67.ec2.internal\" (UID: \"644f10f3f902f0c1cbb90a43f1a164ef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-67.ec2.internal" Jul 29 20:25:05.310475 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.310425 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/644f10f3f902f0c1cbb90a43f1a164ef-config\") pod \"kube-apiserver-proxy-ip-10-0-138-67.ec2.internal\" (UID: \"644f10f3f902f0c1cbb90a43f1a164ef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-67.ec2.internal" Jul 29 20:25:05.403104 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:05.403038 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-67.ec2.internal\" not found" Jul 29 20:25:05.453244 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.453202 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-67.ec2.internal" Jul 29 20:25:05.455740 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.455718 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-67.ec2.internal" Jul 29 20:25:05.504044 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:05.504019 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-67.ec2.internal\" not found" Jul 29 20:25:05.610573 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:05.610514 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-67.ec2.internal\" not found" Jul 29 20:25:05.711712 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:05.711679 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-67.ec2.internal\" not found" Jul 29 20:25:05.750626 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.750595 2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 29 20:25:05.751207 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.750645 2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 20:25:05.751207 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.750730 2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 29 20:25:05.811769 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:05.811738 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-67.ec2.internal\" not found" Jul 29 20:25:05.828540 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.828511 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 20:20:04 +0000 UTC" deadline="2028-01-01 19:22:36.07859925 +0000 UTC" Jul 29 20:25:05.828645 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:05.828537 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12502h57m30.250065495s" Jul 29 20:25:05.912323 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:05.912231 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-67.ec2.internal\" not found" Jul 29 20:25:06.012413 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:06.012383 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-67.ec2.internal\" not found" Jul 29 20:25:06.076345 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:06.076299 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod644f10f3f902f0c1cbb90a43f1a164ef.slice/crio-a96d89444684d069e7a5d03fa458082527714e61e224ef9d2262d49a09707c10 WatchSource:0}: Error finding container a96d89444684d069e7a5d03fa458082527714e61e224ef9d2262d49a09707c10: Status 404 returned error can't find the container with id a96d89444684d069e7a5d03fa458082527714e61e224ef9d2262d49a09707c10 Jul 29 20:25:06.076769 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:06.076743 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeebf028fb5bb404136025b827cf76e9e.slice/crio-ce13a1a4cc8f111586835cbafe05dd75a50f8ff4bb60b4baa16addc4eb07caf9 WatchSource:0}: Error finding container ce13a1a4cc8f111586835cbafe05dd75a50f8ff4bb60b4baa16addc4eb07caf9: Status 404 returned error can't find the container with id ce13a1a4cc8f111586835cbafe05dd75a50f8ff4bb60b4baa16addc4eb07caf9 Jul 29 20:25:06.082192 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.082175 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 20:25:06.113258 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:06.113234 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-67.ec2.internal\" not found" Jul 29 20:25:06.214029 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:06.213956 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-67.ec2.internal\" not found" Jul 29 20:25:06.296874 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.296852 2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 20:25:06.314042 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:06.314013 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-67.ec2.internal\" not found" Jul 29 20:25:06.368381 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.368355 2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 20:25:06.407741 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.407702 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-67.ec2.internal" Jul 29 20:25:06.418704 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.418681 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 29 20:25:06.419869 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.419855 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-67.ec2.internal" Jul 29 20:25:06.432819 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.432803 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 29 20:25:06.785967 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.785938 2559 apiserver.go:52] "Watching apiserver" Jul 29 20:25:06.805394 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.805362 2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 29 20:25:06.807331 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.807303 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-nv7t7","openshift-multus/network-metrics-daemon-vsmj9","openshift-network-operator/iptables-alerter-sql2k","openshift-ovn-kubernetes/ovnkube-node-b46zc","kube-system/global-pull-secret-syncer-sv4n2","kube-system/konnectivity-agent-rbkc4","kube-system/kube-apiserver-proxy-ip-10-0-138-67.ec2.internal","openshift-cluster-node-tuning-operator/tuned-pb67b","openshift-dns/node-resolver-d7c5v","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-67.ec2.internal","openshift-multus/multus-phbfq","openshift-network-diagnostics/network-check-target-n5546","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt","openshift-image-registry/node-ca-zmsn6"] Jul 29 20:25:06.807793 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.807774 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 29 20:25:06.810172 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.810152 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:06.812795 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.812771 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:06.812901 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:06.812848 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsmj9" podUID="e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1" Jul 29 20:25:06.813014 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.812931 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-shf7z\"" Jul 29 20:25:06.813014 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.812948 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 29 20:25:06.813519 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.813260 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 29 20:25:06.813519 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.813282 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 29 20:25:06.814928 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.814907 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sql2k" Jul 29 20:25:06.818142 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.818121 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:06.818981 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.818596 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mxl9m\"" Jul 29 20:25:06.818981 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.818765 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 29 20:25:06.818981 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.818868 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 29 20:25:06.819202 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.819053 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 29 20:25:06.820223 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.820203 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:06.820313 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:06.820272 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4n2" podUID="b26164a7-1242-459e-8152-4a15c188d00e" Jul 29 20:25:06.820376 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.820361 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5shzx\" (UniqueName: \"kubernetes.io/projected/4052a0f7-56d9-4e17-a0d5-117b705d857b-kube-api-access-5shzx\") pod \"aws-ebs-csi-driver-node-xmxwt\" (UID: \"4052a0f7-56d9-4e17-a0d5-117b705d857b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:06.820430 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.820391 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4052a0f7-56d9-4e17-a0d5-117b705d857b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xmxwt\" (UID: \"4052a0f7-56d9-4e17-a0d5-117b705d857b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:06.820430 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.820416 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4052a0f7-56d9-4e17-a0d5-117b705d857b-registration-dir\") pod \"aws-ebs-csi-driver-node-xmxwt\" (UID: \"4052a0f7-56d9-4e17-a0d5-117b705d857b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:06.820532 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.820440 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4052a0f7-56d9-4e17-a0d5-117b705d857b-etc-selinux\") pod \"aws-ebs-csi-driver-node-xmxwt\" (UID: \"4052a0f7-56d9-4e17-a0d5-117b705d857b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:06.820532 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.820468 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4052a0f7-56d9-4e17-a0d5-117b705d857b-socket-dir\") pod \"aws-ebs-csi-driver-node-xmxwt\" (UID: \"4052a0f7-56d9-4e17-a0d5-117b705d857b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:06.820532 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.820492 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4052a0f7-56d9-4e17-a0d5-117b705d857b-device-dir\") pod \"aws-ebs-csi-driver-node-xmxwt\" (UID: \"4052a0f7-56d9-4e17-a0d5-117b705d857b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:06.820532 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.820515 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4052a0f7-56d9-4e17-a0d5-117b705d857b-sys-fs\") pod \"aws-ebs-csi-driver-node-xmxwt\" (UID: \"4052a0f7-56d9-4e17-a0d5-117b705d857b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:06.823189 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.823165 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rbkc4" Jul 29 20:25:06.824294 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.824271 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 29 20:25:06.825093 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.824477 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 29 20:25:06.825093 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.824522 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 29 20:25:06.825093 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.824480 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 29 20:25:06.825093 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.824709 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 29 20:25:06.825093 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.824711 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 29 20:25:06.825093 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.824806 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8sxln\"" Jul 29 20:25:06.825093 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.824999 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 29 20:25:06.825770 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.825447 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-b5l2g\"" Jul 29 20:25:06.825770 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.825687 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 29 20:25:06.826353 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.825871 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 29 20:25:06.827852 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.827557 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:06.829972 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.829916 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 29 20:25:06.830210 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.830190 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:06.830687 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.830663 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 29 20:25:06.830687 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.830672 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 29 20:25:06.830811 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.830672 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 29 20:25:06.831478 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.831456 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 29 20:25:06.831641 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.831625 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ldwqz\"" Jul 29 20:25:06.833492 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.833474 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 29 20:25:06.833660 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.833646 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 29 20:25:06.833729 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.833672 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xr5b6\"" Jul 29 20:25:06.833795 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.833780 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-d7c5v" Jul 29 20:25:06.835992 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.835970 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zmsn6" Jul 29 20:25:06.839286 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.839265 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6slhk\"" Jul 29 20:25:06.839464 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.839442 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 29 20:25:06.839934 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.839911 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 29 20:25:06.840159 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.840140 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 29 20:25:06.840316 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.840295 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 29 20:25:06.840446 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.840428 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 29 20:25:06.840856 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.840833 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-phbfq" Jul 29 20:25:06.841356 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.841337 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-s4w5v\"" Jul 29 20:25:06.843682 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.843664 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 29 20:25:06.843958 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.843938 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lfp9q\"" Jul 29 20:25:06.844573 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.844551 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:06.844690 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:06.844667 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5546" podUID="ae6d959b-d6ab-4669-a6ac-0db9bafde846" Jul 29 20:25:06.846467 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.846446 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-z6sfx" Jul 29 20:25:06.855869 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.855844 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-z6sfx" Jul 29 20:25:06.909584 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.909558 2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 29 20:25:06.921684 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.921651 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-etc-sysconfig\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:06.921818 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.921695 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-run\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:06.921818 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.921725 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kpt6\" (UniqueName: \"kubernetes.io/projected/0518f888-370e-4524-a09d-1e2321611351-kube-api-access-6kpt6\") pod \"iptables-alerter-sql2k\" (UID: \"0518f888-370e-4524-a09d-1e2321611351\") " pod="openshift-network-operator/iptables-alerter-sql2k" Jul 29 20:25:06.921818 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.921751 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c5ec56df-1bb7-42be-846b-17648ead013f-serviceca\") pod \"node-ca-zmsn6\" (UID: \"c5ec56df-1bb7-42be-846b-17648ead013f\") " pod="openshift-image-registry/node-ca-zmsn6" Jul 29 20:25:06.921818 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.921780 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4052a0f7-56d9-4e17-a0d5-117b705d857b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xmxwt\" (UID: \"4052a0f7-56d9-4e17-a0d5-117b705d857b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:06.922145 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.921841 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kdbq\" (UniqueName: \"kubernetes.io/projected/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-kube-api-access-4kdbq\") pod \"network-metrics-daemon-vsmj9\" (UID: \"e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1\") " pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:06.922145 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.921868 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42d531a1-1dba-44b6-a335-7c32dc39dc4f-system-cni-dir\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:06.922145 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.921902 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-run-systemd\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:06.922145 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.921925 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4052a0f7-56d9-4e17-a0d5-117b705d857b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xmxwt\" (UID: \"4052a0f7-56d9-4e17-a0d5-117b705d857b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:06.922145 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.921933 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-run-ovn\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:06.922145 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.921989 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-node-log\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:06.922145 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922013 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/42288536-6377-4883-90bb-69df7fc92db3-hosts-file\") pod \"node-resolver-d7c5v\" (UID: \"42288536-6377-4883-90bb-69df7fc92db3\") " pod="openshift-dns/node-resolver-d7c5v" Jul 29 20:25:06.922145 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922038 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-hostroot\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:06.922145 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922071 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4052a0f7-56d9-4e17-a0d5-117b705d857b-socket-dir\") pod \"aws-ebs-csi-driver-node-xmxwt\" (UID: \"4052a0f7-56d9-4e17-a0d5-117b705d857b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:06.922597 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922150 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5shzx\" (UniqueName: \"kubernetes.io/projected/4052a0f7-56d9-4e17-a0d5-117b705d857b-kube-api-access-5shzx\") pod \"aws-ebs-csi-driver-node-xmxwt\" (UID: \"4052a0f7-56d9-4e17-a0d5-117b705d857b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:06.922597 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922177 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b26164a7-1242-459e-8152-4a15c188d00e-kubelet-config\") pod \"global-pull-secret-syncer-sv4n2\" (UID: \"b26164a7-1242-459e-8152-4a15c188d00e\") " pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:06.922597 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922202 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d837c888-2f72-4f39-8338-da84d1b30057-agent-certs\") pod \"konnectivity-agent-rbkc4\" (UID: \"d837c888-2f72-4f39-8338-da84d1b30057\") " pod="kube-system/konnectivity-agent-rbkc4" Jul 29 20:25:06.922597 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922224 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-sys\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:06.922597 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922249 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-var-lib-openvswitch\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:06.922597 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922267 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4052a0f7-56d9-4e17-a0d5-117b705d857b-socket-dir\") pod \"aws-ebs-csi-driver-node-xmxwt\" (UID: \"4052a0f7-56d9-4e17-a0d5-117b705d857b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:06.922597 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922284 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dhnq\" (UniqueName: \"kubernetes.io/projected/c5ec56df-1bb7-42be-846b-17648ead013f-kube-api-access-7dhnq\") pod \"node-ca-zmsn6\" (UID: \"c5ec56df-1bb7-42be-846b-17648ead013f\") " pod="openshift-image-registry/node-ca-zmsn6" Jul 29 20:25:06.922597 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922315 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4052a0f7-56d9-4e17-a0d5-117b705d857b-registration-dir\") pod \"aws-ebs-csi-driver-node-xmxwt\" (UID: \"4052a0f7-56d9-4e17-a0d5-117b705d857b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:06.922597 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922364 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-log-socket\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:06.922597 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922387 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42d531a1-1dba-44b6-a335-7c32dc39dc4f-cnibin\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:06.922597 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922389 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4052a0f7-56d9-4e17-a0d5-117b705d857b-registration-dir\") pod \"aws-ebs-csi-driver-node-xmxwt\" (UID: \"4052a0f7-56d9-4e17-a0d5-117b705d857b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:06.922597 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922414 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42d531a1-1dba-44b6-a335-7c32dc39dc4f-cni-binary-copy\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:06.922597 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922531 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-host\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:06.922597 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922582 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e432495-2e2e-4265-887e-55187e5f877d-tmp\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:06.923307 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922609 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-host-var-lib-kubelet\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:06.923307 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922634 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpjnw\" (UniqueName: \"kubernetes.io/projected/278e281a-76db-41fb-8cef-3cdbd6a94701-kube-api-access-gpjnw\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:06.923307 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922658 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret\") pod \"global-pull-secret-syncer-sv4n2\" (UID: \"b26164a7-1242-459e-8152-4a15c188d00e\") " pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:06.923307 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922685 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-ovnkube-script-lib\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:06.923307 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922716 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jh6k\" (UniqueName: \"kubernetes.io/projected/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-kube-api-access-2jh6k\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:06.923307 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922755 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w8jq\" (UniqueName: \"kubernetes.io/projected/42288536-6377-4883-90bb-69df7fc92db3-kube-api-access-6w8jq\") pod \"node-resolver-d7c5v\" (UID: \"42288536-6377-4883-90bb-69df7fc92db3\") " pod="openshift-dns/node-resolver-d7c5v" Jul 29 20:25:06.923307 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922790 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-multus-socket-dir-parent\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:06.923307 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922837 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-multus-conf-dir\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:06.923307 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922870 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-etc-systemd\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:06.923307 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922900 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-host-kubelet\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:06.923307 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922925 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:06.923307 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922958 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-host-run-multus-certs\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:06.923307 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.922985 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-etc-kubernetes\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:06.923307 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923009 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-os-release\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:06.923307 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923038 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzm5f\" (UniqueName: \"kubernetes.io/projected/42d531a1-1dba-44b6-a335-7c32dc39dc4f-kube-api-access-pzm5f\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:06.923307 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923083 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-etc-modprobe-d\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:06.924070 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923110 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-host-cni-netd\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:06.924070 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923142 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-ovnkube-config\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:06.924070 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923164 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5ec56df-1bb7-42be-846b-17648ead013f-host\") pod \"node-ca-zmsn6\" (UID: \"c5ec56df-1bb7-42be-846b-17648ead013f\") " pod="openshift-image-registry/node-ca-zmsn6" Jul 29 20:25:06.924070 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923188 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-multus-cni-dir\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:06.924070 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923211 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-cnibin\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:06.924070 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923244 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs\") pod \"network-metrics-daemon-vsmj9\" (UID: \"e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1\") " pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:06.924070 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923267 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42d531a1-1dba-44b6-a335-7c32dc39dc4f-os-release\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:06.924070 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923299 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3e432495-2e2e-4265-887e-55187e5f877d-etc-tuned\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:06.924070 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923323 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0518f888-370e-4524-a09d-1e2321611351-host-slash\") pod \"iptables-alerter-sql2k\" (UID: \"0518f888-370e-4524-a09d-1e2321611351\") " pod="openshift-network-operator/iptables-alerter-sql2k" Jul 29 20:25:06.924070 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923348 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-host-run-netns\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:06.924070 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923373 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-host-run-k8s-cni-cncf-io\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:06.924070 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923459 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-etc-sysctl-conf\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:06.924070 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923489 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-ovn-node-metrics-cert\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:06.924070 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923518 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4052a0f7-56d9-4e17-a0d5-117b705d857b-etc-selinux\") pod \"aws-ebs-csi-driver-node-xmxwt\" (UID: \"4052a0f7-56d9-4e17-a0d5-117b705d857b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:06.924070 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923555 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/42d531a1-1dba-44b6-a335-7c32dc39dc4f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:06.924070 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923580 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0518f888-370e-4524-a09d-1e2321611351-iptables-alerter-script\") pod \"iptables-alerter-sql2k\" (UID: \"0518f888-370e-4524-a09d-1e2321611351\") " pod="openshift-network-operator/iptables-alerter-sql2k" Jul 29 20:25:06.924893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923605 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-etc-openvswitch\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:06.924893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923629 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-host-cni-bin\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:06.924893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923704 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-env-overrides\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:06.924893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923730 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-host-run-netns\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:06.924893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923754 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-etc-sysctl-d\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:06.924893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923782 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-run-openvswitch\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:06.924893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923808 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/278e281a-76db-41fb-8cef-3cdbd6a94701-cni-binary-copy\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:06.924893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923834 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-host-var-lib-cni-bin\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:06.924893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923864 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d837c888-2f72-4f39-8338-da84d1b30057-konnectivity-ca\") pod \"konnectivity-agent-rbkc4\" (UID: \"d837c888-2f72-4f39-8338-da84d1b30057\") " pod="kube-system/konnectivity-agent-rbkc4" Jul 29 20:25:06.924893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923890 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42d531a1-1dba-44b6-a335-7c32dc39dc4f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:06.924893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.923982 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-var-lib-kubelet\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:06.924893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.924011 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgntt\" (UniqueName: \"kubernetes.io/projected/3e432495-2e2e-4265-887e-55187e5f877d-kube-api-access-cgntt\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:06.924893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.924036 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42288536-6377-4883-90bb-69df7fc92db3-tmp-dir\") pod \"node-resolver-d7c5v\" (UID: \"42288536-6377-4883-90bb-69df7fc92db3\") " pod="openshift-dns/node-resolver-d7c5v" Jul 29 20:25:06.924893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.924067 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-host-var-lib-cni-multus\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:06.924893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.924091 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/278e281a-76db-41fb-8cef-3cdbd6a94701-multus-daemon-config\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:06.924893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.924117 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-systemd-units\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:06.924893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.924159 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-host-slash\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:06.925660 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.924185 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b26164a7-1242-459e-8152-4a15c188d00e-dbus\") pod \"global-pull-secret-syncer-sv4n2\" (UID: \"b26164a7-1242-459e-8152-4a15c188d00e\") " pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:06.925660 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.924210 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-lib-modules\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:06.925660 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.924236 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-host-run-ovn-kubernetes\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:06.925660 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.924265 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-system-cni-dir\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:06.925660 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.924291 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-etc-kubernetes\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:06.925660 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.924316 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b24vr\" (UniqueName: \"kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr\") pod \"network-check-target-n5546\" (UID: \"ae6d959b-d6ab-4669-a6ac-0db9bafde846\") " pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:06.925660 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.924347 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4052a0f7-56d9-4e17-a0d5-117b705d857b-device-dir\") pod \"aws-ebs-csi-driver-node-xmxwt\" (UID: \"4052a0f7-56d9-4e17-a0d5-117b705d857b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:06.925660 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.924374 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4052a0f7-56d9-4e17-a0d5-117b705d857b-sys-fs\") pod \"aws-ebs-csi-driver-node-xmxwt\" (UID: \"4052a0f7-56d9-4e17-a0d5-117b705d857b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:06.925660 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.924503 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/42d531a1-1dba-44b6-a335-7c32dc39dc4f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:06.925660 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.924603 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4052a0f7-56d9-4e17-a0d5-117b705d857b-sys-fs\") pod \"aws-ebs-csi-driver-node-xmxwt\" (UID: \"4052a0f7-56d9-4e17-a0d5-117b705d857b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:06.925660 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.924712 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4052a0f7-56d9-4e17-a0d5-117b705d857b-etc-selinux\") pod \"aws-ebs-csi-driver-node-xmxwt\" (UID: \"4052a0f7-56d9-4e17-a0d5-117b705d857b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:06.925660 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.924804 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4052a0f7-56d9-4e17-a0d5-117b705d857b-device-dir\") pod \"aws-ebs-csi-driver-node-xmxwt\" (UID: \"4052a0f7-56d9-4e17-a0d5-117b705d857b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:06.930665 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.930563 2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 29 20:25:06.936524 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:06.936482 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5shzx\" (UniqueName: \"kubernetes.io/projected/4052a0f7-56d9-4e17-a0d5-117b705d857b-kube-api-access-5shzx\") pod \"aws-ebs-csi-driver-node-xmxwt\" (UID: \"4052a0f7-56d9-4e17-a0d5-117b705d857b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:07.024323 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.024267 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-67.ec2.internal" event={"ID":"644f10f3f902f0c1cbb90a43f1a164ef","Type":"ContainerStarted","Data":"a96d89444684d069e7a5d03fa458082527714e61e224ef9d2262d49a09707c10"} Jul 29 20:25:07.027368 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.024785 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b26164a7-1242-459e-8152-4a15c188d00e-dbus\") pod \"global-pull-secret-syncer-sv4n2\" (UID: \"b26164a7-1242-459e-8152-4a15c188d00e\") " pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:07.027368 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.024821 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-lib-modules\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.027368 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.024847 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-host-run-ovn-kubernetes\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.027368 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.024864 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-system-cni-dir\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.027368 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.024880 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-etc-kubernetes\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.027368 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.024922 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b24vr\" (UniqueName: \"kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr\") pod \"network-check-target-n5546\" (UID: \"ae6d959b-d6ab-4669-a6ac-0db9bafde846\") " pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:07.027368 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.024947 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/42d531a1-1dba-44b6-a335-7c32dc39dc4f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:07.027368 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.024963 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-etc-sysconfig\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.027368 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.024987 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-run\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.027368 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025002 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kpt6\" (UniqueName: \"kubernetes.io/projected/0518f888-370e-4524-a09d-1e2321611351-kube-api-access-6kpt6\") pod \"iptables-alerter-sql2k\" (UID: \"0518f888-370e-4524-a09d-1e2321611351\") " pod="openshift-network-operator/iptables-alerter-sql2k" Jul 29 20:25:07.027368 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025017 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c5ec56df-1bb7-42be-846b-17648ead013f-serviceca\") pod \"node-ca-zmsn6\" (UID: \"c5ec56df-1bb7-42be-846b-17648ead013f\") " pod="openshift-image-registry/node-ca-zmsn6" Jul 29 20:25:07.027368 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025037 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kdbq\" (UniqueName: \"kubernetes.io/projected/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-kube-api-access-4kdbq\") pod \"network-metrics-daemon-vsmj9\" (UID: \"e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1\") " pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:07.027368 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025052 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42d531a1-1dba-44b6-a335-7c32dc39dc4f-system-cni-dir\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:07.027368 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025076 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-run-systemd\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.027368 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025120 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-run-ovn\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.027368 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025143 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-node-log\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.027368 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025164 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/42288536-6377-4883-90bb-69df7fc92db3-hosts-file\") pod \"node-resolver-d7c5v\" (UID: \"42288536-6377-4883-90bb-69df7fc92db3\") " pod="openshift-dns/node-resolver-d7c5v" Jul 29 20:25:07.028269 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025186 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-hostroot\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.028269 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025239 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b26164a7-1242-459e-8152-4a15c188d00e-kubelet-config\") pod \"global-pull-secret-syncer-sv4n2\" (UID: \"b26164a7-1242-459e-8152-4a15c188d00e\") " pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:07.028269 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025261 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d837c888-2f72-4f39-8338-da84d1b30057-agent-certs\") pod \"konnectivity-agent-rbkc4\" (UID: \"d837c888-2f72-4f39-8338-da84d1b30057\") " pod="kube-system/konnectivity-agent-rbkc4" Jul 29 20:25:07.028269 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025282 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-sys\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.028269 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025306 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-var-lib-openvswitch\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.028269 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025343 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dhnq\" (UniqueName: \"kubernetes.io/projected/c5ec56df-1bb7-42be-846b-17648ead013f-kube-api-access-7dhnq\") pod \"node-ca-zmsn6\" (UID: \"c5ec56df-1bb7-42be-846b-17648ead013f\") " pod="openshift-image-registry/node-ca-zmsn6" Jul 29 20:25:07.028269 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025377 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-log-socket\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.028269 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025400 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42d531a1-1dba-44b6-a335-7c32dc39dc4f-cnibin\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:07.028269 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025424 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42d531a1-1dba-44b6-a335-7c32dc39dc4f-cni-binary-copy\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:07.028269 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025450 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-host\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.028269 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025456 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-host-run-ovn-kubernetes\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.028269 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025472 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e432495-2e2e-4265-887e-55187e5f877d-tmp\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.028269 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025507 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-host-var-lib-kubelet\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.028269 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025538 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpjnw\" (UniqueName: \"kubernetes.io/projected/278e281a-76db-41fb-8cef-3cdbd6a94701-kube-api-access-gpjnw\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.028269 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025565 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret\") pod \"global-pull-secret-syncer-sv4n2\" (UID: \"b26164a7-1242-459e-8152-4a15c188d00e\") " pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:07.028269 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025590 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-ovnkube-script-lib\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.028269 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025626 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jh6k\" (UniqueName: \"kubernetes.io/projected/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-kube-api-access-2jh6k\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.029108 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025658 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w8jq\" (UniqueName: \"kubernetes.io/projected/42288536-6377-4883-90bb-69df7fc92db3-kube-api-access-6w8jq\") pod \"node-resolver-d7c5v\" (UID: \"42288536-6377-4883-90bb-69df7fc92db3\") " pod="openshift-dns/node-resolver-d7c5v" Jul 29 20:25:07.029108 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025702 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-multus-socket-dir-parent\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.029108 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025705 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b26164a7-1242-459e-8152-4a15c188d00e-dbus\") pod \"global-pull-secret-syncer-sv4n2\" (UID: \"b26164a7-1242-459e-8152-4a15c188d00e\") " pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:07.029108 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025728 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-multus-conf-dir\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.029108 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025748 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/42d531a1-1dba-44b6-a335-7c32dc39dc4f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:07.029108 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025762 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-etc-systemd\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.029108 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025784 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-host-kubelet\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.029108 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025806 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-lib-modules\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.029108 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025835 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.029108 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025861 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-host-run-multus-certs\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.029108 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025924 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-etc-kubernetes\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.029108 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025951 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-os-release\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.029108 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.025987 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzm5f\" (UniqueName: \"kubernetes.io/projected/42d531a1-1dba-44b6-a335-7c32dc39dc4f-kube-api-access-pzm5f\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:07.029108 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026004 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-system-cni-dir\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.029108 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026013 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-etc-modprobe-d\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.029108 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026013 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-log-socket\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.029108 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026057 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-etc-kubernetes\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.029992 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026065 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-host-cni-netd\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.029992 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026085 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-etc-sysconfig\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.029992 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026108 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-ovnkube-config\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.029992 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026131 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5ec56df-1bb7-42be-846b-17648ead013f-host\") pod \"node-ca-zmsn6\" (UID: \"c5ec56df-1bb7-42be-846b-17648ead013f\") " pod="openshift-image-registry/node-ca-zmsn6" Jul 29 20:25:07.029992 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026156 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-multus-cni-dir\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.029992 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026160 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-run\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.029992 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026213 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-cnibin\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.029992 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026268 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs\") pod \"network-metrics-daemon-vsmj9\" (UID: \"e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1\") " pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:07.029992 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026297 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42d531a1-1dba-44b6-a335-7c32dc39dc4f-os-release\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:07.029992 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026321 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3e432495-2e2e-4265-887e-55187e5f877d-etc-tuned\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.029992 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026344 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0518f888-370e-4524-a09d-1e2321611351-host-slash\") pod \"iptables-alerter-sql2k\" (UID: \"0518f888-370e-4524-a09d-1e2321611351\") " pod="openshift-network-operator/iptables-alerter-sql2k" Jul 29 20:25:07.029992 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026405 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-host-run-netns\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.029992 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026429 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-host-run-k8s-cni-cncf-io\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.029992 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026440 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-67.ec2.internal" event={"ID":"eebf028fb5bb404136025b827cf76e9e","Type":"ContainerStarted","Data":"ce13a1a4cc8f111586835cbafe05dd75a50f8ff4bb60b4baa16addc4eb07caf9"} Jul 29 20:25:07.029992 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026465 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-etc-sysctl-conf\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.029992 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026499 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-ovn-node-metrics-cert\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.029992 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026517 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/42288536-6377-4883-90bb-69df7fc92db3-hosts-file\") pod \"node-resolver-d7c5v\" (UID: \"42288536-6377-4883-90bb-69df7fc92db3\") " pod="openshift-dns/node-resolver-d7c5v" Jul 29 20:25:07.029992 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026544 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/42d531a1-1dba-44b6-a335-7c32dc39dc4f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:07.030908 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026606 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0518f888-370e-4524-a09d-1e2321611351-iptables-alerter-script\") pod \"iptables-alerter-sql2k\" (UID: \"0518f888-370e-4524-a09d-1e2321611351\") " pod="openshift-network-operator/iptables-alerter-sql2k" Jul 29 20:25:07.030908 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026692 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-etc-openvswitch\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.030908 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026737 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-host-cni-bin\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.030908 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026761 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-env-overrides\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.030908 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026785 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-host-run-netns\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.030908 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026809 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-etc-sysctl-d\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.030908 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026834 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-run-openvswitch\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.030908 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026874 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/278e281a-76db-41fb-8cef-3cdbd6a94701-cni-binary-copy\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.030908 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026898 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-host-var-lib-cni-bin\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.030908 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026932 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d837c888-2f72-4f39-8338-da84d1b30057-konnectivity-ca\") pod \"konnectivity-agent-rbkc4\" (UID: \"d837c888-2f72-4f39-8338-da84d1b30057\") " pod="kube-system/konnectivity-agent-rbkc4" Jul 29 20:25:07.030908 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026952 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42d531a1-1dba-44b6-a335-7c32dc39dc4f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:07.030908 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026973 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-var-lib-kubelet\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.030908 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026984 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c5ec56df-1bb7-42be-846b-17648ead013f-serviceca\") pod \"node-ca-zmsn6\" (UID: \"c5ec56df-1bb7-42be-846b-17648ead013f\") " pod="openshift-image-registry/node-ca-zmsn6" Jul 29 20:25:07.030908 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.027044 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42d531a1-1dba-44b6-a335-7c32dc39dc4f-cnibin\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:07.030908 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.026991 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgntt\" (UniqueName: \"kubernetes.io/projected/3e432495-2e2e-4265-887e-55187e5f877d-kube-api-access-cgntt\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.030908 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.027241 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-node-log\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.030908 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.027309 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42288536-6377-4883-90bb-69df7fc92db3-tmp-dir\") pod \"node-resolver-d7c5v\" (UID: \"42288536-6377-4883-90bb-69df7fc92db3\") " pod="openshift-dns/node-resolver-d7c5v" Jul 29 20:25:07.030908 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.028284 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-sys\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.031815 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.028340 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-var-lib-openvswitch\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.031815 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.028409 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42d531a1-1dba-44b6-a335-7c32dc39dc4f-cni-binary-copy\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:07.031815 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.028489 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42d531a1-1dba-44b6-a335-7c32dc39dc4f-system-cni-dir\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:07.031815 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.028597 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-run-systemd\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.031815 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.028643 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-etc-systemd\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.031815 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.028689 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-hostroot\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.031815 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.028708 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b26164a7-1242-459e-8152-4a15c188d00e-kubelet-config\") pod \"global-pull-secret-syncer-sv4n2\" (UID: \"b26164a7-1242-459e-8152-4a15c188d00e\") " pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:07.031815 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.028739 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-run-ovn\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.031815 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.028791 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42288536-6377-4883-90bb-69df7fc92db3-tmp-dir\") pod \"node-resolver-d7c5v\" (UID: \"42288536-6377-4883-90bb-69df7fc92db3\") " pod="openshift-dns/node-resolver-d7c5v" Jul 29 20:25:07.031815 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.028798 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-host-var-lib-cni-multus\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.031815 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.028812 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-multus-cni-dir\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.031815 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.028842 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/278e281a-76db-41fb-8cef-3cdbd6a94701-multus-daemon-config\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.031815 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.028880 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-systemd-units\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.031815 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.028913 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-host-slash\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.031815 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.029026 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 20:25:07.031815 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.029085 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-host-slash\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.031815 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.029158 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-host-var-lib-cni-multus\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.031815 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.029709 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-systemd-units\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.032906 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.029772 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-host-kubelet\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.032906 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.029882 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.032906 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.029915 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-etc-modprobe-d\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.032906 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.029958 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-host-run-multus-certs\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.032906 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.029953 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-host-cni-netd\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.032906 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.030068 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5ec56df-1bb7-42be-846b-17648ead013f-host\") pod \"node-ca-zmsn6\" (UID: \"c5ec56df-1bb7-42be-846b-17648ead013f\") " pod="openshift-image-registry/node-ca-zmsn6" Jul 29 20:25:07.032906 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.030069 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-os-release\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.032906 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.030159 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-etc-kubernetes\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.032906 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.030193 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-host\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.032906 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.030289 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-multus-socket-dir-parent\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.032906 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.030790 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d837c888-2f72-4f39-8338-da84d1b30057-agent-certs\") pod \"konnectivity-agent-rbkc4\" (UID: \"d837c888-2f72-4f39-8338-da84d1b30057\") " pod="kube-system/konnectivity-agent-rbkc4" Jul 29 20:25:07.032906 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.030883 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-ovnkube-script-lib\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.032906 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.031310 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-host-var-lib-kubelet\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.032906 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.031472 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-multus-conf-dir\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.032906 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.031479 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0518f888-370e-4524-a09d-1e2321611351-iptables-alerter-script\") pod \"iptables-alerter-sql2k\" (UID: \"0518f888-370e-4524-a09d-1e2321611351\") " pod="openshift-network-operator/iptables-alerter-sql2k" Jul 29 20:25:07.032906 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.031593 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-cnibin\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.032906 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.031594 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-etc-openvswitch\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.032906 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.031641 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-host-cni-bin\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.033905 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:07.031720 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 20:25:07.033905 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:07.031833 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs podName:e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1 nodeName:}" failed. No retries permitted until 2026-07-29 20:25:07.531790652 +0000 UTC m=+3.047096861 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs") pod "network-metrics-daemon-vsmj9" (UID: "e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 20:25:07.033905 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.031932 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/278e281a-76db-41fb-8cef-3cdbd6a94701-multus-daemon-config\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.033905 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.032034 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-host-run-netns\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.033905 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.032065 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-env-overrides\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.033905 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.032139 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42d531a1-1dba-44b6-a335-7c32dc39dc4f-os-release\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:07.033905 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.032633 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d837c888-2f72-4f39-8338-da84d1b30057-konnectivity-ca\") pod \"konnectivity-agent-rbkc4\" (UID: \"d837c888-2f72-4f39-8338-da84d1b30057\") " pod="kube-system/konnectivity-agent-rbkc4" Jul 29 20:25:07.033905 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.032662 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/42d531a1-1dba-44b6-a335-7c32dc39dc4f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:07.033905 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:07.032768 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 20:25:07.033905 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.032779 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-etc-sysctl-d\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.033905 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.032881 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-run-openvswitch\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.033905 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:07.032893 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret podName:b26164a7-1242-459e-8152-4a15c188d00e nodeName:}" failed. No retries permitted until 2026-07-29 20:25:07.532875449 +0000 UTC m=+3.048181670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret") pod "global-pull-secret-syncer-sv4n2" (UID: "b26164a7-1242-459e-8152-4a15c188d00e") : object "kube-system"/"original-pull-secret" not registered Jul 29 20:25:07.033905 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.032934 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e432495-2e2e-4265-887e-55187e5f877d-tmp\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.033905 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.032987 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-host-var-lib-cni-bin\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.033905 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.033108 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-host-run-netns\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.033905 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.033451 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/278e281a-76db-41fb-8cef-3cdbd6a94701-cni-binary-copy\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.033905 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.033575 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/278e281a-76db-41fb-8cef-3cdbd6a94701-host-run-k8s-cni-cncf-io\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.034876 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.033637 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0518f888-370e-4524-a09d-1e2321611351-host-slash\") pod \"iptables-alerter-sql2k\" (UID: \"0518f888-370e-4524-a09d-1e2321611351\") " pod="openshift-network-operator/iptables-alerter-sql2k" Jul 29 20:25:07.034876 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.033791 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42d531a1-1dba-44b6-a335-7c32dc39dc4f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:07.034876 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.033873 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-var-lib-kubelet\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.034876 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.034041 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3e432495-2e2e-4265-887e-55187e5f877d-etc-sysctl-conf\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.034876 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.034552 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-ovn-node-metrics-cert\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.035151 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.035106 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-ovnkube-config\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.035857 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.035831 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3e432495-2e2e-4265-887e-55187e5f877d-etc-tuned\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.038158 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:07.038068 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 20:25:07.038158 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:07.038104 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 20:25:07.038158 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:07.038118 2559 projected.go:196] Error preparing data for projected volume kube-api-access-b24vr for pod openshift-network-diagnostics/network-check-target-n5546: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 20:25:07.038413 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:07.038198 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr podName:ae6d959b-d6ab-4669-a6ac-0db9bafde846 nodeName:}" failed. No retries permitted until 2026-07-29 20:25:07.53818305 +0000 UTC m=+3.053489270 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b24vr" (UniqueName: "kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr") pod "network-check-target-n5546" (UID: "ae6d959b-d6ab-4669-a6ac-0db9bafde846") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 20:25:07.040808 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.040755 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgntt\" (UniqueName: \"kubernetes.io/projected/3e432495-2e2e-4265-887e-55187e5f877d-kube-api-access-cgntt\") pod \"tuned-pb67b\" (UID: \"3e432495-2e2e-4265-887e-55187e5f877d\") " pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.042096 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.041696 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kdbq\" (UniqueName: \"kubernetes.io/projected/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-kube-api-access-4kdbq\") pod \"network-metrics-daemon-vsmj9\" (UID: \"e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1\") " pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:07.042817 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.042790 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kpt6\" (UniqueName: \"kubernetes.io/projected/0518f888-370e-4524-a09d-1e2321611351-kube-api-access-6kpt6\") pod \"iptables-alerter-sql2k\" (UID: \"0518f888-370e-4524-a09d-1e2321611351\") " pod="openshift-network-operator/iptables-alerter-sql2k" Jul 29 20:25:07.043284 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.043257 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzm5f\" (UniqueName: \"kubernetes.io/projected/42d531a1-1dba-44b6-a335-7c32dc39dc4f-kube-api-access-pzm5f\") pod \"multus-additional-cni-plugins-nv7t7\" (UID: \"42d531a1-1dba-44b6-a335-7c32dc39dc4f\") " pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:07.043910 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.043889 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w8jq\" (UniqueName: \"kubernetes.io/projected/42288536-6377-4883-90bb-69df7fc92db3-kube-api-access-6w8jq\") pod \"node-resolver-d7c5v\" (UID: \"42288536-6377-4883-90bb-69df7fc92db3\") " pod="openshift-dns/node-resolver-d7c5v" Jul 29 20:25:07.044096 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.044069 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jh6k\" (UniqueName: \"kubernetes.io/projected/6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553-kube-api-access-2jh6k\") pod \"ovnkube-node-b46zc\" (UID: \"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553\") " pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.044718 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.044691 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dhnq\" (UniqueName: \"kubernetes.io/projected/c5ec56df-1bb7-42be-846b-17648ead013f-kube-api-access-7dhnq\") pod \"node-ca-zmsn6\" (UID: \"c5ec56df-1bb7-42be-846b-17648ead013f\") " pod="openshift-image-registry/node-ca-zmsn6" Jul 29 20:25:07.045396 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.045376 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpjnw\" (UniqueName: \"kubernetes.io/projected/278e281a-76db-41fb-8cef-3cdbd6a94701-kube-api-access-gpjnw\") pod \"multus-phbfq\" (UID: \"278e281a-76db-41fb-8cef-3cdbd6a94701\") " pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.125626 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.125478 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" Jul 29 20:25:07.139832 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.139811 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sql2k" Jul 29 20:25:07.148675 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.148645 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:07.154541 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.154521 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rbkc4" Jul 29 20:25:07.161348 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.161323 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nv7t7" Jul 29 20:25:07.170021 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.169994 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pb67b" Jul 29 20:25:07.178761 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.178670 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-d7c5v" Jul 29 20:25:07.187413 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.187393 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zmsn6" Jul 29 20:25:07.193994 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.193976 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-phbfq" Jul 29 20:25:07.532407 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.532327 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs\") pod \"network-metrics-daemon-vsmj9\" (UID: \"e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1\") " pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:07.532562 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:07.532438 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 20:25:07.532562 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:07.532499 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs podName:e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1 nodeName:}" failed. No retries permitted until 2026-07-29 20:25:08.532479544 +0000 UTC m=+4.047785763 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs") pod "network-metrics-daemon-vsmj9" (UID: "e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 20:25:07.632801 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.632765 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b24vr\" (UniqueName: \"kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr\") pod \"network-check-target-n5546\" (UID: \"ae6d959b-d6ab-4669-a6ac-0db9bafde846\") " pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:07.632954 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.632833 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret\") pod \"global-pull-secret-syncer-sv4n2\" (UID: \"b26164a7-1242-459e-8152-4a15c188d00e\") " pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:07.632954 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:07.632925 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 20:25:07.632954 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:07.632948 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 20:25:07.633105 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:07.632960 2559 projected.go:196] Error preparing data for projected volume kube-api-access-b24vr for pod openshift-network-diagnostics/network-check-target-n5546: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 20:25:07.633105 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:07.632965 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 20:25:07.633105 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:07.633027 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret podName:b26164a7-1242-459e-8152-4a15c188d00e nodeName:}" failed. No retries permitted until 2026-07-29 20:25:08.633008804 +0000 UTC m=+4.148315010 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret") pod "global-pull-secret-syncer-sv4n2" (UID: "b26164a7-1242-459e-8152-4a15c188d00e") : object "kube-system"/"original-pull-secret" not registered Jul 29 20:25:07.633105 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:07.633082 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr podName:ae6d959b-d6ab-4669-a6ac-0db9bafde846 nodeName:}" failed. No retries permitted until 2026-07-29 20:25:08.633064509 +0000 UTC m=+4.148370730 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b24vr" (UniqueName: "kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr") pod "network-check-target-n5546" (UID: "ae6d959b-d6ab-4669-a6ac-0db9bafde846") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 20:25:07.859716 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.859633 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 20:20:06 +0000 UTC" deadline="2028-01-23 04:14:17.84521328 +0000 UTC" Jul 29 20:25:07.859716 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:07.859673 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13015h49m9.985545815s" Jul 29 20:25:08.019383 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:08.019360 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:08.019480 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:08.019462 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4n2" podUID="b26164a7-1242-459e-8152-4a15c188d00e" Jul 29 20:25:08.031044 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:08.031021 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e432495_2e2e_4265_887e_55187e5f877d.slice/crio-acfa976e9bc0bc9d27646496bf295fd4315bbf9f3b97ebcf33cf050c91819e51 WatchSource:0}: Error finding container acfa976e9bc0bc9d27646496bf295fd4315bbf9f3b97ebcf33cf050c91819e51: Status 404 returned error can't find the container with id acfa976e9bc0bc9d27646496bf295fd4315bbf9f3b97ebcf33cf050c91819e51 Jul 29 20:25:08.032548 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:08.032522 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42d531a1_1dba_44b6_a335_7c32dc39dc4f.slice/crio-28ed379b603d08be1b411ee4dfd5359ef6c5b94e7ff8a2dbb1c5bb98e51141be WatchSource:0}: Error finding container 28ed379b603d08be1b411ee4dfd5359ef6c5b94e7ff8a2dbb1c5bb98e51141be: Status 404 returned error can't find the container with id 28ed379b603d08be1b411ee4dfd5359ef6c5b94e7ff8a2dbb1c5bb98e51141be Jul 29 20:25:08.035946 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:08.035826 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cdd0fdc_ea0d_4a7b_a96b_b8067c7bd553.slice/crio-6b9407a809474693708730af31138e215be9f649b33375086774249a6d32fdf6 WatchSource:0}: Error finding container 6b9407a809474693708730af31138e215be9f649b33375086774249a6d32fdf6: Status 404 returned error can't find the container with id 6b9407a809474693708730af31138e215be9f649b33375086774249a6d32fdf6 Jul 29 20:25:08.056909 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:08.056873 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4052a0f7_56d9_4e17_a0d5_117b705d857b.slice/crio-7f7ff80e3d76d9b4ad4d2825627c58d3e6d9101d48099974a9b1fe69cf7ef137 WatchSource:0}: Error finding container 7f7ff80e3d76d9b4ad4d2825627c58d3e6d9101d48099974a9b1fe69cf7ef137: Status 404 returned error can't find the container with id 7f7ff80e3d76d9b4ad4d2825627c58d3e6d9101d48099974a9b1fe69cf7ef137 Jul 29 20:25:08.057695 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:08.057667 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0518f888_370e_4524_a09d_1e2321611351.slice/crio-318e95b782824d6b89d89028e0ffe69401081fce4a9f4917c0319f2418a676d7 WatchSource:0}: Error finding container 318e95b782824d6b89d89028e0ffe69401081fce4a9f4917c0319f2418a676d7: Status 404 returned error can't find the container with id 318e95b782824d6b89d89028e0ffe69401081fce4a9f4917c0319f2418a676d7 Jul 29 20:25:08.058709 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:08.058678 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5ec56df_1bb7_42be_846b_17648ead013f.slice/crio-e39f8b71515d796990be95ce8c1ae0bdba61f1002b0c837ed2a04eb2e8595827 WatchSource:0}: Error finding container e39f8b71515d796990be95ce8c1ae0bdba61f1002b0c837ed2a04eb2e8595827: Status 404 returned error can't find the container with id e39f8b71515d796990be95ce8c1ae0bdba61f1002b0c837ed2a04eb2e8595827 Jul 29 20:25:08.060269 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:08.060245 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd837c888_2f72_4f39_8338_da84d1b30057.slice/crio-d313351e26ac437f1bd096fad6850ead4e697776749dfe14ed9b8b44dff885b2 WatchSource:0}: Error finding container d313351e26ac437f1bd096fad6850ead4e697776749dfe14ed9b8b44dff885b2: Status 404 returned error can't find the container with id d313351e26ac437f1bd096fad6850ead4e697776749dfe14ed9b8b44dff885b2 Jul 29 20:25:08.540662 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:08.540308 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs\") pod \"network-metrics-daemon-vsmj9\" (UID: \"e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1\") " pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:08.540662 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:08.540569 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 20:25:08.540662 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:08.540645 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs podName:e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1 nodeName:}" failed. No retries permitted until 2026-07-29 20:25:10.540624287 +0000 UTC m=+6.055930502 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs") pod "network-metrics-daemon-vsmj9" (UID: "e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 20:25:08.640951 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:08.640911 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b24vr\" (UniqueName: \"kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr\") pod \"network-check-target-n5546\" (UID: \"ae6d959b-d6ab-4669-a6ac-0db9bafde846\") " pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:08.641145 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:08.640981 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret\") pod \"global-pull-secret-syncer-sv4n2\" (UID: \"b26164a7-1242-459e-8152-4a15c188d00e\") " pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:08.641145 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:08.641130 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 20:25:08.641249 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:08.641191 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret podName:b26164a7-1242-459e-8152-4a15c188d00e nodeName:}" failed. No retries permitted until 2026-07-29 20:25:10.641174235 +0000 UTC m=+6.156480450 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret") pod "global-pull-secret-syncer-sv4n2" (UID: "b26164a7-1242-459e-8152-4a15c188d00e") : object "kube-system"/"original-pull-secret" not registered Jul 29 20:25:08.641675 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:08.641604 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 20:25:08.641675 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:08.641628 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 20:25:08.641675 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:08.641640 2559 projected.go:196] Error preparing data for projected volume kube-api-access-b24vr for pod openshift-network-diagnostics/network-check-target-n5546: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 20:25:08.641874 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:08.641681 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr podName:ae6d959b-d6ab-4669-a6ac-0db9bafde846 nodeName:}" failed. No retries permitted until 2026-07-29 20:25:10.641668555 +0000 UTC m=+6.156974757 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-b24vr" (UniqueName: "kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr") pod "network-check-target-n5546" (UID: "ae6d959b-d6ab-4669-a6ac-0db9bafde846") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 20:25:08.860077 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:08.859923 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 20:20:06 +0000 UTC" deadline="2028-01-21 12:45:36.323588333 +0000 UTC" Jul 29 20:25:08.860077 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:08.859985 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12976h20m27.463607884s" Jul 29 20:25:09.020681 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:09.020178 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:09.020681 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:09.020319 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsmj9" podUID="e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1" Jul 29 20:25:09.021136 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:09.020982 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:09.021136 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:09.021086 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5546" podUID="ae6d959b-d6ab-4669-a6ac-0db9bafde846" Jul 29 20:25:09.046493 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:09.046460 2559 generic.go:356] "Generic (PLEG): container finished" podID="eebf028fb5bb404136025b827cf76e9e" containerID="7623d76b4d71d844e9125200017557f35dc70482a00cce9a6b62bc09365058e0" exitCode=0 Jul 29 20:25:09.046640 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:09.046544 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-67.ec2.internal" event={"ID":"eebf028fb5bb404136025b827cf76e9e","Type":"ContainerDied","Data":"7623d76b4d71d844e9125200017557f35dc70482a00cce9a6b62bc09365058e0"} Jul 29 20:25:09.052070 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:09.051665 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sql2k" event={"ID":"0518f888-370e-4524-a09d-1e2321611351","Type":"ContainerStarted","Data":"318e95b782824d6b89d89028e0ffe69401081fce4a9f4917c0319f2418a676d7"} Jul 29 20:25:09.057536 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:09.057460 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" event={"ID":"4052a0f7-56d9-4e17-a0d5-117b705d857b","Type":"ContainerStarted","Data":"7f7ff80e3d76d9b4ad4d2825627c58d3e6d9101d48099974a9b1fe69cf7ef137"} Jul 29 20:25:09.060452 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:09.060427 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-phbfq" event={"ID":"278e281a-76db-41fb-8cef-3cdbd6a94701","Type":"ContainerStarted","Data":"f89b0601f05407fe84ed518e4f8d7b85fe2194c682c63951d595340cf0c7f613"} Jul 29 20:25:09.066510 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:09.066487 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-d7c5v" event={"ID":"42288536-6377-4883-90bb-69df7fc92db3","Type":"ContainerStarted","Data":"75d1ffc5b62af01ac379deb329fca6858e6db4899a3eae18322187fbb4156e7a"} Jul 29 20:25:09.074478 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:09.074239 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nv7t7" event={"ID":"42d531a1-1dba-44b6-a335-7c32dc39dc4f","Type":"ContainerStarted","Data":"28ed379b603d08be1b411ee4dfd5359ef6c5b94e7ff8a2dbb1c5bb98e51141be"} Jul 29 20:25:09.095366 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:09.095337 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-67.ec2.internal" event={"ID":"644f10f3f902f0c1cbb90a43f1a164ef","Type":"ContainerStarted","Data":"fb7cbac62a2c52e667dd252b403c6019154960e289f2e32a5d901286303231e8"} Jul 29 20:25:09.106178 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:09.106145 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rbkc4" event={"ID":"d837c888-2f72-4f39-8338-da84d1b30057","Type":"ContainerStarted","Data":"d313351e26ac437f1bd096fad6850ead4e697776749dfe14ed9b8b44dff885b2"} Jul 29 20:25:09.123961 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:09.123886 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zmsn6" event={"ID":"c5ec56df-1bb7-42be-846b-17648ead013f","Type":"ContainerStarted","Data":"e39f8b71515d796990be95ce8c1ae0bdba61f1002b0c837ed2a04eb2e8595827"} Jul 29 20:25:09.130534 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:09.130505 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" event={"ID":"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553","Type":"ContainerStarted","Data":"6b9407a809474693708730af31138e215be9f649b33375086774249a6d32fdf6"} Jul 29 20:25:09.141291 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:09.141219 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pb67b" event={"ID":"3e432495-2e2e-4265-887e-55187e5f877d","Type":"ContainerStarted","Data":"acfa976e9bc0bc9d27646496bf295fd4315bbf9f3b97ebcf33cf050c91819e51"} Jul 29 20:25:10.019697 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:10.019663 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:10.020213 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:10.019804 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4n2" podUID="b26164a7-1242-459e-8152-4a15c188d00e" Jul 29 20:25:10.157230 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:10.157198 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-67.ec2.internal" event={"ID":"eebf028fb5bb404136025b827cf76e9e","Type":"ContainerStarted","Data":"8d42fb81172f63476f3deac09923a0a22179bdf9521365459c997b6079243720"} Jul 29 20:25:10.171975 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:10.171912 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-67.ec2.internal" podStartSLOduration=4.171892259 podStartE2EDuration="4.171892259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:25:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:25:09.111527123 +0000 UTC m=+4.626833350" watchObservedRunningTime="2026-07-29 20:25:10.171892259 +0000 UTC m=+5.687198483" Jul 29 20:25:10.554963 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:10.554925 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs\") pod \"network-metrics-daemon-vsmj9\" (UID: \"e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1\") " pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:10.555185 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:10.555167 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 20:25:10.555251 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:10.555237 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs podName:e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1 nodeName:}" failed. No retries permitted until 2026-07-29 20:25:14.555220008 +0000 UTC m=+10.070526213 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs") pod "network-metrics-daemon-vsmj9" (UID: "e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 20:25:10.656242 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:10.656208 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret\") pod \"global-pull-secret-syncer-sv4n2\" (UID: \"b26164a7-1242-459e-8152-4a15c188d00e\") " pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:10.656411 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:10.656290 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b24vr\" (UniqueName: \"kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr\") pod \"network-check-target-n5546\" (UID: \"ae6d959b-d6ab-4669-a6ac-0db9bafde846\") " pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:10.656477 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:10.656437 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 20:25:10.656477 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:10.656458 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 20:25:10.656477 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:10.656469 2559 projected.go:196] Error preparing data for projected volume kube-api-access-b24vr for pod openshift-network-diagnostics/network-check-target-n5546: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 20:25:10.656609 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:10.656528 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr podName:ae6d959b-d6ab-4669-a6ac-0db9bafde846 nodeName:}" failed. No retries permitted until 2026-07-29 20:25:14.656509436 +0000 UTC m=+10.171815659 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-b24vr" (UniqueName: "kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr") pod "network-check-target-n5546" (UID: "ae6d959b-d6ab-4669-a6ac-0db9bafde846") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 20:25:10.656947 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:10.656914 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 20:25:10.657079 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:10.656968 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret podName:b26164a7-1242-459e-8152-4a15c188d00e nodeName:}" failed. No retries permitted until 2026-07-29 20:25:14.656952974 +0000 UTC m=+10.172259193 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret") pod "global-pull-secret-syncer-sv4n2" (UID: "b26164a7-1242-459e-8152-4a15c188d00e") : object "kube-system"/"original-pull-secret" not registered Jul 29 20:25:11.025591 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:11.022964 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:11.025591 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:11.023138 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5546" podUID="ae6d959b-d6ab-4669-a6ac-0db9bafde846" Jul 29 20:25:11.025591 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:11.023516 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:11.025591 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:11.023652 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsmj9" podUID="e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1" Jul 29 20:25:12.020547 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:12.020076 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:12.020547 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:12.020209 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4n2" podUID="b26164a7-1242-459e-8152-4a15c188d00e" Jul 29 20:25:13.021618 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:13.021575 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:13.022078 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:13.021690 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5546" podUID="ae6d959b-d6ab-4669-a6ac-0db9bafde846" Jul 29 20:25:13.022078 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:13.021762 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:13.022078 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:13.021845 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsmj9" podUID="e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1" Jul 29 20:25:14.019370 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:14.019337 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:14.019516 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:14.019472 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4n2" podUID="b26164a7-1242-459e-8152-4a15c188d00e" Jul 29 20:25:14.589263 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:14.589227 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs\") pod \"network-metrics-daemon-vsmj9\" (UID: \"e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1\") " pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:14.589659 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:14.589399 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 20:25:14.589659 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:14.589462 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs podName:e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1 nodeName:}" failed. No retries permitted until 2026-07-29 20:25:22.589441444 +0000 UTC m=+18.104747659 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs") pod "network-metrics-daemon-vsmj9" (UID: "e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 20:25:14.689799 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:14.689761 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret\") pod \"global-pull-secret-syncer-sv4n2\" (UID: \"b26164a7-1242-459e-8152-4a15c188d00e\") " pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:14.689949 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:14.689854 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b24vr\" (UniqueName: \"kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr\") pod \"network-check-target-n5546\" (UID: \"ae6d959b-d6ab-4669-a6ac-0db9bafde846\") " pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:14.690014 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:14.689998 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 20:25:14.690085 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:14.690020 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 20:25:14.690085 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:14.690033 2559 projected.go:196] Error preparing data for projected volume kube-api-access-b24vr for pod openshift-network-diagnostics/network-check-target-n5546: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 20:25:14.690198 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:14.690107 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr podName:ae6d959b-d6ab-4669-a6ac-0db9bafde846 nodeName:}" failed. No retries permitted until 2026-07-29 20:25:22.690088964 +0000 UTC m=+18.205395168 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-b24vr" (UniqueName: "kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr") pod "network-check-target-n5546" (UID: "ae6d959b-d6ab-4669-a6ac-0db9bafde846") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 20:25:14.690487 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:14.690470 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 20:25:14.690542 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:14.690521 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret podName:b26164a7-1242-459e-8152-4a15c188d00e nodeName:}" failed. No retries permitted until 2026-07-29 20:25:22.690506738 +0000 UTC m=+18.205812942 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret") pod "global-pull-secret-syncer-sv4n2" (UID: "b26164a7-1242-459e-8152-4a15c188d00e") : object "kube-system"/"original-pull-secret" not registered Jul 29 20:25:15.021612 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:15.021533 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:15.021770 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:15.021702 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsmj9" podUID="e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1" Jul 29 20:25:15.023365 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:15.022136 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:15.023365 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:15.022283 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5546" podUID="ae6d959b-d6ab-4669-a6ac-0db9bafde846" Jul 29 20:25:16.020314 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:16.019835 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:16.020314 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:16.019957 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4n2" podUID="b26164a7-1242-459e-8152-4a15c188d00e" Jul 29 20:25:17.021292 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:17.021261 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:17.021675 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:17.021375 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5546" podUID="ae6d959b-d6ab-4669-a6ac-0db9bafde846" Jul 29 20:25:17.021732 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:17.021720 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:17.021835 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:17.021814 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsmj9" podUID="e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1" Jul 29 20:25:18.019690 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:18.019653 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:18.019861 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:18.019773 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4n2" podUID="b26164a7-1242-459e-8152-4a15c188d00e" Jul 29 20:25:19.019459 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:19.019419 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:19.019856 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:19.019576 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsmj9" podUID="e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1" Jul 29 20:25:19.019856 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:19.019419 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:19.019856 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:19.019676 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5546" podUID="ae6d959b-d6ab-4669-a6ac-0db9bafde846" Jul 29 20:25:20.019558 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:20.019521 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:20.019962 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:20.019635 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4n2" podUID="b26164a7-1242-459e-8152-4a15c188d00e" Jul 29 20:25:21.019461 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:21.019423 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:21.019636 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:21.019560 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5546" podUID="ae6d959b-d6ab-4669-a6ac-0db9bafde846" Jul 29 20:25:21.019636 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:21.019591 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:21.020016 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:21.019661 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsmj9" podUID="e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1" Jul 29 20:25:22.019350 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:22.019315 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:22.019594 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:22.019435 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4n2" podUID="b26164a7-1242-459e-8152-4a15c188d00e" Jul 29 20:25:22.652843 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:22.652808 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs\") pod \"network-metrics-daemon-vsmj9\" (UID: \"e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1\") " pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:22.653312 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:22.652935 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 20:25:22.653312 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:22.652999 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs podName:e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1 nodeName:}" failed. No retries permitted until 2026-07-29 20:25:38.652978485 +0000 UTC m=+34.168284706 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs") pod "network-metrics-daemon-vsmj9" (UID: "e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 20:25:22.753405 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:22.753369 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret\") pod \"global-pull-secret-syncer-sv4n2\" (UID: \"b26164a7-1242-459e-8152-4a15c188d00e\") " pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:22.753575 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:22.753457 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b24vr\" (UniqueName: \"kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr\") pod \"network-check-target-n5546\" (UID: \"ae6d959b-d6ab-4669-a6ac-0db9bafde846\") " pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:22.753575 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:22.753526 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 20:25:22.753677 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:22.753578 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 20:25:22.753677 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:22.753593 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret podName:b26164a7-1242-459e-8152-4a15c188d00e nodeName:}" failed. No retries permitted until 2026-07-29 20:25:38.753575119 +0000 UTC m=+34.268881339 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret") pod "global-pull-secret-syncer-sv4n2" (UID: "b26164a7-1242-459e-8152-4a15c188d00e") : object "kube-system"/"original-pull-secret" not registered Jul 29 20:25:22.753677 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:22.753596 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 20:25:22.753677 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:22.753607 2559 projected.go:196] Error preparing data for projected volume kube-api-access-b24vr for pod openshift-network-diagnostics/network-check-target-n5546: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 20:25:22.753677 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:22.753640 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr podName:ae6d959b-d6ab-4669-a6ac-0db9bafde846 nodeName:}" failed. No retries permitted until 2026-07-29 20:25:38.753629848 +0000 UTC m=+34.268936054 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-b24vr" (UniqueName: "kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr") pod "network-check-target-n5546" (UID: "ae6d959b-d6ab-4669-a6ac-0db9bafde846") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 20:25:23.019707 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:23.019588 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:23.019874 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:23.019717 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5546" podUID="ae6d959b-d6ab-4669-a6ac-0db9bafde846" Jul 29 20:25:23.019874 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:23.019780 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:23.019981 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:23.019888 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsmj9" podUID="e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1" Jul 29 20:25:24.019695 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:24.019658 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:24.020226 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:24.019793 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4n2" podUID="b26164a7-1242-459e-8152-4a15c188d00e" Jul 29 20:25:25.020127 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:25.020087 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:25.020127 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:25.020107 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:25.020549 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:25.020210 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5546" podUID="ae6d959b-d6ab-4669-a6ac-0db9bafde846" Jul 29 20:25:25.020549 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:25.020342 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsmj9" podUID="e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1" Jul 29 20:25:26.019690 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:26.019647 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:26.019851 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:26.019767 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4n2" podUID="b26164a7-1242-459e-8152-4a15c188d00e" Jul 29 20:25:26.194832 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:26.194802 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pb67b" event={"ID":"3e432495-2e2e-4265-887e-55187e5f877d","Type":"ContainerStarted","Data":"528bdbb22705331eabd4f0008a858aca66ff0f3cea495e6a239e80d62653f89f"} Jul 29 20:25:26.217036 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:26.216984 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pb67b" podStartSLOduration=3.170493148 podStartE2EDuration="21.216966205s" totalImagesPullingTime="18.046473057s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:25:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:25:26.216534097 +0000 UTC m=+21.731840323" watchObservedRunningTime="2026-07-29 20:25:26.216966205 +0000 UTC m=+21.732272433" Jul 29 20:25:26.217223 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:26.217191 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-67.ec2.internal" podStartSLOduration=20.217180879 podStartE2EDuration="20.217180879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:25:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:25:10.173005658 +0000 UTC m=+5.688311883" watchObservedRunningTime="2026-07-29 20:25:26.217180879 +0000 UTC m=+21.732487104" Jul 29 20:25:27.019845 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.019667 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:27.020093 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.019670 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:27.020093 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:27.019924 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5546" podUID="ae6d959b-d6ab-4669-a6ac-0db9bafde846" Jul 29 20:25:27.020093 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:27.020001 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsmj9" podUID="e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1" Jul 29 20:25:27.198281 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.198244 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zmsn6" event={"ID":"c5ec56df-1bb7-42be-846b-17648ead013f","Type":"ContainerStarted","Data":"52328d0ce228b145ef7f2689cbbc9d74459706ebb492a57f641d15568797f438"} Jul 29 20:25:27.201043 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.201014 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" event={"ID":"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553","Type":"ContainerStarted","Data":"da8e346178fd5a2875b1c5608895054ff997fca35c7420832bc4148b75e46796"} Jul 29 20:25:27.201175 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.201101 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" event={"ID":"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553","Type":"ContainerStarted","Data":"1c5882b7aac06bf32eb2062cd8c0481bb955114c94b8d478d6f05f49776b3a4a"} Jul 29 20:25:27.201175 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.201132 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" event={"ID":"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553","Type":"ContainerStarted","Data":"4a3ce5f05ccd732c0fa26c9ea40bee452fe8a7b3ae1fbe3dfa0a3b0de16ef374"} Jul 29 20:25:27.201175 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.201164 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" event={"ID":"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553","Type":"ContainerStarted","Data":"dca2a340230937ba56104be6a2628b61978fc826b20e297599b14460e42c0592"} Jul 29 20:25:27.201276 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.201268 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" event={"ID":"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553","Type":"ContainerStarted","Data":"082b1cd4a1b49c383dd9d3dc1fe5da385ef0b491f6b28358fc7c74979609b96e"} Jul 29 20:25:27.201351 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.201335 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" event={"ID":"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553","Type":"ContainerStarted","Data":"b7c139f1ecfc58f3c0c9074138a19bcf166200ac6763cd8d4b6639754d14d181"} Jul 29 20:25:27.202390 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.202372 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" event={"ID":"4052a0f7-56d9-4e17-a0d5-117b705d857b","Type":"ContainerStarted","Data":"8d7e753297f36d8534ee13f856eced1bfdf64caa08e66e72face1a19bb5daaf4"} Jul 29 20:25:27.203799 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.203770 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-phbfq" event={"ID":"278e281a-76db-41fb-8cef-3cdbd6a94701","Type":"ContainerStarted","Data":"15268022f526638c965a9abb4e6192c4451f2477690f00d6de7e3e712e75b457"} Jul 29 20:25:27.205011 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.204986 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-d7c5v" event={"ID":"42288536-6377-4883-90bb-69df7fc92db3","Type":"ContainerStarted","Data":"5ba37e619f39ef1350647c52ef4db8448ff2aa1915be308e5531075a930ec7a9"} Jul 29 20:25:27.206254 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.206232 2559 generic.go:356] "Generic (PLEG): container finished" podID="42d531a1-1dba-44b6-a335-7c32dc39dc4f" containerID="376ba1c238d5e1bfe5d318955dd162d87ad32d8d3ec5ee779307068df8dedcce" exitCode=0 Jul 29 20:25:27.206346 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.206292 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nv7t7" event={"ID":"42d531a1-1dba-44b6-a335-7c32dc39dc4f","Type":"ContainerDied","Data":"376ba1c238d5e1bfe5d318955dd162d87ad32d8d3ec5ee779307068df8dedcce"} Jul 29 20:25:27.207648 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.207578 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rbkc4" event={"ID":"d837c888-2f72-4f39-8338-da84d1b30057","Type":"ContainerStarted","Data":"4feef7a0716328815f9840f6211811c2fa60c54abdfac490999388a6105d96a9"} Jul 29 20:25:27.213655 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.213618 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zmsn6" podStartSLOduration=4.194925053 podStartE2EDuration="22.213606769s" totalImagesPullingTime="18.018681716s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:25:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:25:27.213545436 +0000 UTC m=+22.728851653" watchObservedRunningTime="2026-07-29 20:25:27.213606769 +0000 UTC m=+22.728912992" Jul 29 20:25:27.252344 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.252302 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-d7c5v" podStartSLOduration=4.218347325 podStartE2EDuration="22.252289101s" totalImagesPullingTime="18.033941776s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:25:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:25:27.229387939 +0000 UTC m=+22.744694163" watchObservedRunningTime="2026-07-29 20:25:27.252289101 +0000 UTC m=+22.767595377" Jul 29 20:25:27.272012 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.271974 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-phbfq" podStartSLOduration=4.200104773 podStartE2EDuration="22.271960629s" totalImagesPullingTime="18.071855856s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:25:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:25:27.252751445 +0000 UTC m=+22.768057667" watchObservedRunningTime="2026-07-29 20:25:27.271960629 +0000 UTC m=+22.787266852" Jul 29 20:25:27.295525 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.295489 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rbkc4" podStartSLOduration=4.277876014 podStartE2EDuration="22.295475241s" totalImagesPullingTime="18.017599227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:25:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:25:27.272141505 +0000 UTC m=+22.787447729" watchObservedRunningTime="2026-07-29 20:25:27.295475241 +0000 UTC m=+22.810781463" Jul 29 20:25:27.497966 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.497945 2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 29 20:25:27.886096 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.885976 2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T20:25:27.497960911Z","UUID":"42789d44-cb96-4cc4-926d-3fdfce907e80","Handler":null,"Name":"","Endpoint":""} Jul 29 20:25:27.887852 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.887818 2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 29 20:25:27.887988 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.887860 2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 29 20:25:27.945630 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.945545 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rbkc4" Jul 29 20:25:27.946133 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:27.946111 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rbkc4" Jul 29 20:25:28.019744 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:28.019720 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:28.019961 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:28.019827 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4n2" podUID="b26164a7-1242-459e-8152-4a15c188d00e" Jul 29 20:25:28.222029 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:28.221946 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sql2k" event={"ID":"0518f888-370e-4524-a09d-1e2321611351","Type":"ContainerStarted","Data":"8c08902ee55ddbac84c613dc7556e38d3d395d86b645c530f862a3934716edf3"} Jul 29 20:25:28.224638 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:28.224596 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" event={"ID":"4052a0f7-56d9-4e17-a0d5-117b705d857b","Type":"ContainerStarted","Data":"9e7a2808e490c82933abf0eed43d91af8b3ee5e7e8344577a6161af15c20ed58"} Jul 29 20:25:28.238903 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:28.238854 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sql2k" podStartSLOduration=5.219384537 podStartE2EDuration="23.238837352s" totalImagesPullingTime="18.019452815s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:25:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:25:28.238119007 +0000 UTC m=+23.753425233" watchObservedRunningTime="2026-07-29 20:25:28.238837352 +0000 UTC m=+23.754143576" Jul 29 20:25:29.019591 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:29.019515 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:29.019591 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:29.019522 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:29.019797 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:29.019691 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5546" podUID="ae6d959b-d6ab-4669-a6ac-0db9bafde846" Jul 29 20:25:29.019853 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:29.019796 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsmj9" podUID="e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1" Jul 29 20:25:29.230915 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:29.230879 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" event={"ID":"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553","Type":"ContainerStarted","Data":"342971bb6e77371ba7b9433911438203ebabcdde449149a1c588b56128bb1a40"} Jul 29 20:25:29.233274 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:29.233242 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" event={"ID":"4052a0f7-56d9-4e17-a0d5-117b705d857b","Type":"ContainerStarted","Data":"e7578aaa4b85663ef1497048d711d9a7bff7adf580df87b3e25d2889f4544550"} Jul 29 20:25:29.233415 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:29.233291 2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 29 20:25:30.020223 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:30.020029 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:30.020384 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:30.020298 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4n2" podUID="b26164a7-1242-459e-8152-4a15c188d00e" Jul 29 20:25:30.323378 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:30.323313 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rbkc4" Jul 29 20:25:30.323772 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:30.323425 2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 29 20:25:30.323860 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:30.323843 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rbkc4" Jul 29 20:25:30.340161 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:30.340122 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmxwt" podStartSLOduration=5.201377021 podStartE2EDuration="25.340108589s" totalImagesPullingTime="20.138731568s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:25:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:25:29.267659955 +0000 UTC m=+24.782966177" watchObservedRunningTime="2026-07-29 20:25:30.340108589 +0000 UTC m=+25.855414806" Jul 29 20:25:31.020411 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:31.019890 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:31.020411 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:31.019890 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:31.020411 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:31.020064 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsmj9" podUID="e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1" Jul 29 20:25:31.020411 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:31.020115 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5546" podUID="ae6d959b-d6ab-4669-a6ac-0db9bafde846" Jul 29 20:25:32.019269 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:32.019081 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:32.020093 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:32.019356 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4n2" podUID="b26164a7-1242-459e-8152-4a15c188d00e" Jul 29 20:25:32.241063 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:32.241014 2559 generic.go:356] "Generic (PLEG): container finished" podID="42d531a1-1dba-44b6-a335-7c32dc39dc4f" containerID="55fcb8bb39c2aab6e4f0ec9f67c17ee347c5e960371bdd40bc68a5c066ef1436" exitCode=0 Jul 29 20:25:32.241283 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:32.241087 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nv7t7" event={"ID":"42d531a1-1dba-44b6-a335-7c32dc39dc4f","Type":"ContainerDied","Data":"55fcb8bb39c2aab6e4f0ec9f67c17ee347c5e960371bdd40bc68a5c066ef1436"} Jul 29 20:25:32.244916 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:32.244893 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" event={"ID":"6cdd0fdc-ea0d-4a7b-a96b-b8067c7bd553","Type":"ContainerStarted","Data":"2cb25983ce3787fd762e63a026e393e3eff36af4049b52e38cf7e0c3bcc78ea2"} Jul 29 20:25:32.245166 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:32.245145 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:32.260338 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:32.260319 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:32.288941 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:32.288860 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" podStartSLOduration=9.195125614 podStartE2EDuration="27.288848426s" totalImagesPullingTime="18.093722812s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:25:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:25:32.288799004 +0000 UTC m=+27.804105228" watchObservedRunningTime="2026-07-29 20:25:32.288848426 +0000 UTC m=+27.804154649" Jul 29 20:25:33.020149 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:33.020115 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:33.020541 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:33.020162 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:33.020541 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:33.020245 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsmj9" podUID="e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1" Jul 29 20:25:33.020541 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:33.020382 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5546" podUID="ae6d959b-d6ab-4669-a6ac-0db9bafde846" Jul 29 20:25:33.238108 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:33.238076 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n5546"] Jul 29 20:25:33.242183 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:33.241988 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vsmj9"] Jul 29 20:25:33.243332 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:33.243310 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sv4n2"] Jul 29 20:25:33.243442 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:33.243414 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:33.243516 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:33.243495 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4n2" podUID="b26164a7-1242-459e-8152-4a15c188d00e" Jul 29 20:25:33.248019 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:33.247967 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:33.248019 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:33.247982 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:33.248187 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:33.248115 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsmj9" podUID="e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1" Jul 29 20:25:33.248251 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:33.248194 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5546" podUID="ae6d959b-d6ab-4669-a6ac-0db9bafde846" Jul 29 20:25:33.249445 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:33.248984 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:33.249445 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:33.249078 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:33.267414 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:33.267395 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:25:34.251929 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:34.251894 2559 generic.go:356] "Generic (PLEG): container finished" podID="42d531a1-1dba-44b6-a335-7c32dc39dc4f" containerID="5d5de0ecefe7bc1b5b22afc24fb342dfe69c42842eae79103484fb51bcab69ba" exitCode=0 Jul 29 20:25:34.252430 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:34.251977 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nv7t7" event={"ID":"42d531a1-1dba-44b6-a335-7c32dc39dc4f","Type":"ContainerDied","Data":"5d5de0ecefe7bc1b5b22afc24fb342dfe69c42842eae79103484fb51bcab69ba"} Jul 29 20:25:35.020170 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:35.019988 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:35.020310 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:35.020067 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:35.020310 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:35.020232 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5546" podUID="ae6d959b-d6ab-4669-a6ac-0db9bafde846" Jul 29 20:25:35.020394 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:35.020323 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsmj9" podUID="e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1" Jul 29 20:25:35.020394 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:35.020104 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:35.020469 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:35.020399 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4n2" podUID="b26164a7-1242-459e-8152-4a15c188d00e" Jul 29 20:25:36.258981 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:36.258951 2559 generic.go:356] "Generic (PLEG): container finished" podID="42d531a1-1dba-44b6-a335-7c32dc39dc4f" containerID="7c041b5a0fbdf5106a63ead601340cb849aa1c511b3b6ae4657313d259ccf2c0" exitCode=0 Jul 29 20:25:36.259474 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:36.259018 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nv7t7" event={"ID":"42d531a1-1dba-44b6-a335-7c32dc39dc4f","Type":"ContainerDied","Data":"7c041b5a0fbdf5106a63ead601340cb849aa1c511b3b6ae4657313d259ccf2c0"} Jul 29 20:25:37.020096 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:37.020063 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:37.020096 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:37.020083 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:37.020349 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:37.020075 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:37.020349 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:37.020200 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vsmj9" podUID="e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1" Jul 29 20:25:37.020349 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:37.020276 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5546" podUID="ae6d959b-d6ab-4669-a6ac-0db9bafde846" Jul 29 20:25:37.020505 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:37.020368 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sv4n2" podUID="b26164a7-1242-459e-8152-4a15c188d00e" Jul 29 20:25:38.674028 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:38.673993 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs\") pod \"network-metrics-daemon-vsmj9\" (UID: \"e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1\") " pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:38.674706 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:38.674126 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 20:25:38.674706 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:38.674186 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs podName:e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1 nodeName:}" failed. No retries permitted until 2026-07-29 20:26:10.674171924 +0000 UTC m=+66.189478139 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs") pod "network-metrics-daemon-vsmj9" (UID: "e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 20:25:38.774457 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:38.774425 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b24vr\" (UniqueName: \"kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr\") pod \"network-check-target-n5546\" (UID: \"ae6d959b-d6ab-4669-a6ac-0db9bafde846\") " pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:38.774625 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:38.774475 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret\") pod \"global-pull-secret-syncer-sv4n2\" (UID: \"b26164a7-1242-459e-8152-4a15c188d00e\") " pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:38.774625 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:38.774579 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 20:25:38.774625 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:38.774605 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 20:25:38.774625 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:38.774628 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 20:25:38.774823 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:38.774634 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret podName:b26164a7-1242-459e-8152-4a15c188d00e nodeName:}" failed. No retries permitted until 2026-07-29 20:26:10.774619165 +0000 UTC m=+66.289925367 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret") pod "global-pull-secret-syncer-sv4n2" (UID: "b26164a7-1242-459e-8152-4a15c188d00e") : object "kube-system"/"original-pull-secret" not registered Jul 29 20:25:38.774823 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:38.774640 2559 projected.go:196] Error preparing data for projected volume kube-api-access-b24vr for pod openshift-network-diagnostics/network-check-target-n5546: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 20:25:38.774823 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:38.774689 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr podName:ae6d959b-d6ab-4669-a6ac-0db9bafde846 nodeName:}" failed. No retries permitted until 2026-07-29 20:26:10.774671582 +0000 UTC m=+66.289977803 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-b24vr" (UniqueName: "kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr") pod "network-check-target-n5546" (UID: "ae6d959b-d6ab-4669-a6ac-0db9bafde846") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 20:25:38.851186 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:38.851104 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-67.ec2.internal" event="NodeReady" Jul 29 20:25:38.851350 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:38.851272 2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 29 20:25:38.944139 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:38.944104 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qcswx"] Jul 29 20:25:38.947849 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:38.947822 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qcswx" Jul 29 20:25:38.950402 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:38.950378 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 29 20:25:38.950579 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:38.950449 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cs4sn\"" Jul 29 20:25:38.951165 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:38.950751 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 29 20:25:38.951165 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:38.950991 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 29 20:25:38.951301 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:38.951240 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nlftr"] Jul 29 20:25:38.954477 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:38.954460 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nlftr" Jul 29 20:25:38.956779 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:38.956760 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 29 20:25:38.957269 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:38.957246 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xpqnl\"" Jul 29 20:25:38.957370 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:38.957247 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 29 20:25:38.960309 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:38.960284 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qcswx"] Jul 29 20:25:38.965004 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:38.964424 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nlftr"] Jul 29 20:25:39.019915 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.019877 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:25:39.020118 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.020042 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:25:39.020118 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.020067 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:25:39.022865 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.022690 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 29 20:25:39.023352 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.023332 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 29 20:25:39.023439 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.023396 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mkp42\"" Jul 29 20:25:39.023601 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.023586 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 29 20:25:39.023669 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.023606 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f7tqs\"" Jul 29 20:25:39.023779 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.023764 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 29 20:25:39.077019 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.076987 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a072f2ce-aee2-412d-90cf-9ecc83a109df-config-volume\") pod \"dns-default-nlftr\" (UID: \"a072f2ce-aee2-412d-90cf-9ecc83a109df\") " pod="openshift-dns/dns-default-nlftr" Jul 29 20:25:39.077162 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.077042 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr85k\" (UniqueName: \"kubernetes.io/projected/655beca7-e0da-4b4c-9001-322b63d9f11a-kube-api-access-zr85k\") pod \"ingress-canary-qcswx\" (UID: \"655beca7-e0da-4b4c-9001-322b63d9f11a\") " pod="openshift-ingress-canary/ingress-canary-qcswx" Jul 29 20:25:39.077162 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.077111 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a072f2ce-aee2-412d-90cf-9ecc83a109df-metrics-tls\") pod \"dns-default-nlftr\" (UID: \"a072f2ce-aee2-412d-90cf-9ecc83a109df\") " pod="openshift-dns/dns-default-nlftr" Jul 29 20:25:39.077162 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.077146 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhdd4\" (UniqueName: \"kubernetes.io/projected/a072f2ce-aee2-412d-90cf-9ecc83a109df-kube-api-access-rhdd4\") pod \"dns-default-nlftr\" (UID: \"a072f2ce-aee2-412d-90cf-9ecc83a109df\") " pod="openshift-dns/dns-default-nlftr" Jul 29 20:25:39.077314 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.077168 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/655beca7-e0da-4b4c-9001-322b63d9f11a-cert\") pod \"ingress-canary-qcswx\" (UID: \"655beca7-e0da-4b4c-9001-322b63d9f11a\") " pod="openshift-ingress-canary/ingress-canary-qcswx" Jul 29 20:25:39.077314 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.077185 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a072f2ce-aee2-412d-90cf-9ecc83a109df-tmp-dir\") pod \"dns-default-nlftr\" (UID: \"a072f2ce-aee2-412d-90cf-9ecc83a109df\") " pod="openshift-dns/dns-default-nlftr" Jul 29 20:25:39.178373 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.178296 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/655beca7-e0da-4b4c-9001-322b63d9f11a-cert\") pod \"ingress-canary-qcswx\" (UID: \"655beca7-e0da-4b4c-9001-322b63d9f11a\") " pod="openshift-ingress-canary/ingress-canary-qcswx" Jul 29 20:25:39.178373 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.178334 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a072f2ce-aee2-412d-90cf-9ecc83a109df-tmp-dir\") pod \"dns-default-nlftr\" (UID: \"a072f2ce-aee2-412d-90cf-9ecc83a109df\") " pod="openshift-dns/dns-default-nlftr" Jul 29 20:25:39.178587 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.178384 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a072f2ce-aee2-412d-90cf-9ecc83a109df-config-volume\") pod \"dns-default-nlftr\" (UID: \"a072f2ce-aee2-412d-90cf-9ecc83a109df\") " pod="openshift-dns/dns-default-nlftr" Jul 29 20:25:39.178644 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.178573 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr85k\" (UniqueName: \"kubernetes.io/projected/655beca7-e0da-4b4c-9001-322b63d9f11a-kube-api-access-zr85k\") pod \"ingress-canary-qcswx\" (UID: \"655beca7-e0da-4b4c-9001-322b63d9f11a\") " pod="openshift-ingress-canary/ingress-canary-qcswx" Jul 29 20:25:39.178703 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.178649 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a072f2ce-aee2-412d-90cf-9ecc83a109df-metrics-tls\") pod \"dns-default-nlftr\" (UID: \"a072f2ce-aee2-412d-90cf-9ecc83a109df\") " pod="openshift-dns/dns-default-nlftr" Jul 29 20:25:39.178748 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.178700 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhdd4\" (UniqueName: \"kubernetes.io/projected/a072f2ce-aee2-412d-90cf-9ecc83a109df-kube-api-access-rhdd4\") pod \"dns-default-nlftr\" (UID: \"a072f2ce-aee2-412d-90cf-9ecc83a109df\") " pod="openshift-dns/dns-default-nlftr" Jul 29 20:25:39.178795 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.178741 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a072f2ce-aee2-412d-90cf-9ecc83a109df-tmp-dir\") pod \"dns-default-nlftr\" (UID: \"a072f2ce-aee2-412d-90cf-9ecc83a109df\") " pod="openshift-dns/dns-default-nlftr" Jul 29 20:25:39.178970 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:39.178952 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 20:25:39.179081 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:39.179025 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a072f2ce-aee2-412d-90cf-9ecc83a109df-metrics-tls podName:a072f2ce-aee2-412d-90cf-9ecc83a109df nodeName:}" failed. No retries permitted until 2026-07-29 20:25:39.679006722 +0000 UTC m=+35.194312924 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a072f2ce-aee2-412d-90cf-9ecc83a109df-metrics-tls") pod "dns-default-nlftr" (UID: "a072f2ce-aee2-412d-90cf-9ecc83a109df") : secret "dns-default-metrics-tls" not found Jul 29 20:25:39.179081 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.179035 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a072f2ce-aee2-412d-90cf-9ecc83a109df-config-volume\") pod \"dns-default-nlftr\" (UID: \"a072f2ce-aee2-412d-90cf-9ecc83a109df\") " pod="openshift-dns/dns-default-nlftr" Jul 29 20:25:39.182828 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.182807 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/655beca7-e0da-4b4c-9001-322b63d9f11a-cert\") pod \"ingress-canary-qcswx\" (UID: \"655beca7-e0da-4b4c-9001-322b63d9f11a\") " pod="openshift-ingress-canary/ingress-canary-qcswx" Jul 29 20:25:39.188189 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.187595 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhdd4\" (UniqueName: \"kubernetes.io/projected/a072f2ce-aee2-412d-90cf-9ecc83a109df-kube-api-access-rhdd4\") pod \"dns-default-nlftr\" (UID: \"a072f2ce-aee2-412d-90cf-9ecc83a109df\") " pod="openshift-dns/dns-default-nlftr" Jul 29 20:25:39.188189 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.188139 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr85k\" (UniqueName: \"kubernetes.io/projected/655beca7-e0da-4b4c-9001-322b63d9f11a-kube-api-access-zr85k\") pod \"ingress-canary-qcswx\" (UID: \"655beca7-e0da-4b4c-9001-322b63d9f11a\") " pod="openshift-ingress-canary/ingress-canary-qcswx" Jul 29 20:25:39.263094 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.263042 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qcswx" Jul 29 20:25:39.682277 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:39.682232 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a072f2ce-aee2-412d-90cf-9ecc83a109df-metrics-tls\") pod \"dns-default-nlftr\" (UID: \"a072f2ce-aee2-412d-90cf-9ecc83a109df\") " pod="openshift-dns/dns-default-nlftr" Jul 29 20:25:39.682684 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:39.682411 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 20:25:39.682684 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:39.682477 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a072f2ce-aee2-412d-90cf-9ecc83a109df-metrics-tls podName:a072f2ce-aee2-412d-90cf-9ecc83a109df nodeName:}" failed. No retries permitted until 2026-07-29 20:25:40.682460844 +0000 UTC m=+36.197767047 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a072f2ce-aee2-412d-90cf-9ecc83a109df-metrics-tls") pod "dns-default-nlftr" (UID: "a072f2ce-aee2-412d-90cf-9ecc83a109df") : secret "dns-default-metrics-tls" not found Jul 29 20:25:40.325242 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:40.325215 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qcswx"] Jul 29 20:25:40.328750 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:40.328725 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod655beca7_e0da_4b4c_9001_322b63d9f11a.slice/crio-30636ac48b3a1a4eab8ed0d4f046bf86004194b5a1e2c352b7a11afe8238221d WatchSource:0}: Error finding container 30636ac48b3a1a4eab8ed0d4f046bf86004194b5a1e2c352b7a11afe8238221d: Status 404 returned error can't find the container with id 30636ac48b3a1a4eab8ed0d4f046bf86004194b5a1e2c352b7a11afe8238221d Jul 29 20:25:40.690330 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:40.690295 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a072f2ce-aee2-412d-90cf-9ecc83a109df-metrics-tls\") pod \"dns-default-nlftr\" (UID: \"a072f2ce-aee2-412d-90cf-9ecc83a109df\") " pod="openshift-dns/dns-default-nlftr" Jul 29 20:25:40.690733 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:40.690451 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 20:25:40.690733 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:40.690518 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a072f2ce-aee2-412d-90cf-9ecc83a109df-metrics-tls podName:a072f2ce-aee2-412d-90cf-9ecc83a109df nodeName:}" failed. No retries permitted until 2026-07-29 20:25:42.690501268 +0000 UTC m=+38.205807489 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a072f2ce-aee2-412d-90cf-9ecc83a109df-metrics-tls") pod "dns-default-nlftr" (UID: "a072f2ce-aee2-412d-90cf-9ecc83a109df") : secret "dns-default-metrics-tls" not found Jul 29 20:25:41.275901 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:41.275864 2559 generic.go:356] "Generic (PLEG): container finished" podID="42d531a1-1dba-44b6-a335-7c32dc39dc4f" containerID="69761bedabdd139c2485ca9f4953a49f54f39c736ba8ee90a2d3d531707e6582" exitCode=0 Jul 29 20:25:41.276081 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:41.275961 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nv7t7" event={"ID":"42d531a1-1dba-44b6-a335-7c32dc39dc4f","Type":"ContainerDied","Data":"69761bedabdd139c2485ca9f4953a49f54f39c736ba8ee90a2d3d531707e6582"} Jul 29 20:25:41.278151 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:41.278120 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qcswx" event={"ID":"655beca7-e0da-4b4c-9001-322b63d9f11a","Type":"ContainerStarted","Data":"30636ac48b3a1a4eab8ed0d4f046bf86004194b5a1e2c352b7a11afe8238221d"} Jul 29 20:25:42.283775 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:42.283737 2559 generic.go:356] "Generic (PLEG): container finished" podID="42d531a1-1dba-44b6-a335-7c32dc39dc4f" containerID="933b38421976c787425f444666766fae31d3cdef23c609cca4bd60c2c37f5d8f" exitCode=0 Jul 29 20:25:42.284250 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:42.283812 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nv7t7" event={"ID":"42d531a1-1dba-44b6-a335-7c32dc39dc4f","Type":"ContainerDied","Data":"933b38421976c787425f444666766fae31d3cdef23c609cca4bd60c2c37f5d8f"} Jul 29 20:25:42.707397 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:42.707302 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a072f2ce-aee2-412d-90cf-9ecc83a109df-metrics-tls\") pod \"dns-default-nlftr\" (UID: \"a072f2ce-aee2-412d-90cf-9ecc83a109df\") " pod="openshift-dns/dns-default-nlftr" Jul 29 20:25:42.707565 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:42.707448 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 20:25:42.707565 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:42.707515 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a072f2ce-aee2-412d-90cf-9ecc83a109df-metrics-tls podName:a072f2ce-aee2-412d-90cf-9ecc83a109df nodeName:}" failed. No retries permitted until 2026-07-29 20:25:46.707496098 +0000 UTC m=+42.222802310 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a072f2ce-aee2-412d-90cf-9ecc83a109df-metrics-tls") pod "dns-default-nlftr" (UID: "a072f2ce-aee2-412d-90cf-9ecc83a109df") : secret "dns-default-metrics-tls" not found Jul 29 20:25:43.290223 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:43.289953 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nv7t7" event={"ID":"42d531a1-1dba-44b6-a335-7c32dc39dc4f","Type":"ContainerStarted","Data":"fc3b6d4e5008da6c8efcada7653b399e58e5eab159c8f44956ea68609922f0fb"} Jul 29 20:25:43.315203 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:43.315040 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nv7t7" podStartSLOduration=9.961409219 podStartE2EDuration="38.315014069s" totalImagesPullingTime="28.35360485s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:25:05 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:25:43.313543313 +0000 UTC m=+38.828849539" watchObservedRunningTime="2026-07-29 20:25:43.315014069 +0000 UTC m=+38.830320294" Jul 29 20:25:44.293638 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:44.293600 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qcswx" event={"ID":"655beca7-e0da-4b4c-9001-322b63d9f11a","Type":"ContainerStarted","Data":"8b64a8b0e9fe3e7d5b6a67e65807a5cad10e9c9e7f7b780546d7c7bb4d87b310"} Jul 29 20:25:44.312870 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:44.312817 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qcswx" podStartSLOduration=3.344065184 podStartE2EDuration="6.312800768s" totalImagesPullingTime="2.968735584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:25:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:25:44.311548537 +0000 UTC m=+39.826854761" watchObservedRunningTime="2026-07-29 20:25:44.312800768 +0000 UTC m=+39.828107031" Jul 29 20:25:46.733676 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:46.733629 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a072f2ce-aee2-412d-90cf-9ecc83a109df-metrics-tls\") pod \"dns-default-nlftr\" (UID: \"a072f2ce-aee2-412d-90cf-9ecc83a109df\") " pod="openshift-dns/dns-default-nlftr" Jul 29 20:25:46.734214 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:46.733776 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 20:25:46.734214 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:46.733848 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a072f2ce-aee2-412d-90cf-9ecc83a109df-metrics-tls podName:a072f2ce-aee2-412d-90cf-9ecc83a109df nodeName:}" failed. No retries permitted until 2026-07-29 20:25:54.733826242 +0000 UTC m=+50.249132444 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a072f2ce-aee2-412d-90cf-9ecc83a109df-metrics-tls") pod "dns-default-nlftr" (UID: "a072f2ce-aee2-412d-90cf-9ecc83a109df") : secret "dns-default-metrics-tls" not found Jul 29 20:25:54.786453 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:54.786411 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a072f2ce-aee2-412d-90cf-9ecc83a109df-metrics-tls\") pod \"dns-default-nlftr\" (UID: \"a072f2ce-aee2-412d-90cf-9ecc83a109df\") " pod="openshift-dns/dns-default-nlftr" Jul 29 20:25:54.786930 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:54.786557 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 20:25:54.786930 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:54.786617 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a072f2ce-aee2-412d-90cf-9ecc83a109df-metrics-tls podName:a072f2ce-aee2-412d-90cf-9ecc83a109df nodeName:}" failed. No retries permitted until 2026-07-29 20:26:10.786602612 +0000 UTC m=+66.301908814 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a072f2ce-aee2-412d-90cf-9ecc83a109df-metrics-tls") pod "dns-default-nlftr" (UID: "a072f2ce-aee2-412d-90cf-9ecc83a109df") : secret "dns-default-metrics-tls" not found Jul 29 20:25:55.718781 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.718750 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt"] Jul 29 20:25:55.728758 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.728732 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-555697c5b6-ljftx"] Jul 29 20:25:55.728881 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.728853 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt" Jul 29 20:25:55.731722 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.731697 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 29 20:25:55.733915 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.733893 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9qlzx\"" Jul 29 20:25:55.734037 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.733944 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 29 20:25:55.734037 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.733974 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 29 20:25:55.734214 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.734040 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 29 20:25:55.737359 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.737333 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt"] Jul 29 20:25:55.737458 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.737447 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:55.739686 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.739665 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-555697c5b6-ljftx"] Jul 29 20:25:55.740143 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.740126 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-brdv4\"" Jul 29 20:25:55.740239 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.740172 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 29 20:25:55.740873 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.740850 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 29 20:25:55.740966 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.740952 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 29 20:25:55.744733 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.744713 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 29 20:25:55.811186 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.811158 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-s7t8m"] Jul 29 20:25:55.827574 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.827542 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-s7t8m" Jul 29 20:25:55.830657 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.830588 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 29 20:25:55.830951 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.830924 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 29 20:25:55.831174 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.830912 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 29 20:25:55.831271 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.831251 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-cnzhr\"" Jul 29 20:25:55.831417 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.831398 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 29 20:25:55.832596 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.832548 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-s7t8m"] Jul 29 20:25:55.843799 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.843778 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 29 20:25:55.894753 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.894719 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-certificates\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:55.894933 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.894775 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kltv\" (UniqueName: \"kubernetes.io/projected/6a729414-a205-4b68-bee8-0c24d964a385-kube-api-access-9kltv\") pod \"cluster-monitoring-operator-548b489cb8-sgttt\" (UID: \"6a729414-a205-4b68-bee8-0c24d964a385\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt" Jul 29 20:25:55.894933 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.894824 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7fa264d5-fcc9-4972-b37e-18fc7697e367-ca-trust-extracted\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:55.894933 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.894845 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7fa264d5-fcc9-4972-b37e-18fc7697e367-installation-pull-secrets\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:55.894933 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.894874 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7fa264d5-fcc9-4972-b37e-18fc7697e367-image-registry-private-configuration\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:55.894933 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.894914 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a729414-a205-4b68-bee8-0c24d964a385-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sgttt\" (UID: \"6a729414-a205-4b68-bee8-0c24d964a385\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt" Jul 29 20:25:55.894933 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.894931 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7fa264d5-fcc9-4972-b37e-18fc7697e367-trusted-ca\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:55.895183 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.894953 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-bound-sa-token\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:55.895183 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.894975 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk4vw\" (UniqueName: \"kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-kube-api-access-fk4vw\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:55.895183 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.894998 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6a729414-a205-4b68-bee8-0c24d964a385-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-sgttt\" (UID: \"6a729414-a205-4b68-bee8-0c24d964a385\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt" Jul 29 20:25:55.895183 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.895013 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-tls\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:55.904147 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.904121 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-42vmx"] Jul 29 20:25:55.913612 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.913593 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-42vmx" Jul 29 20:25:55.916274 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.916247 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 29 20:25:55.916431 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.916341 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-btlpb\"" Jul 29 20:25:55.916509 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.916492 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 29 20:25:55.917360 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.917341 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 29 20:25:55.917416 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.917402 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 29 20:25:55.921204 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.921182 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-42vmx"] Jul 29 20:25:55.995520 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.995492 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc9b1faf-5638-4fd3-ae81-d579fbf2f56f-tmp\") pod \"insights-operator-5885fdb759-s7t8m\" (UID: \"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f\") " pod="openshift-insights/insights-operator-5885fdb759-s7t8m" Jul 29 20:25:55.995681 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.995531 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6a729414-a205-4b68-bee8-0c24d964a385-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-sgttt\" (UID: \"6a729414-a205-4b68-bee8-0c24d964a385\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt" Jul 29 20:25:55.995681 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.995559 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-tls\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:55.995681 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.995608 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-certificates\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:55.995681 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.995626 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fc9b1faf-5638-4fd3-ae81-d579fbf2f56f-snapshots\") pod \"insights-operator-5885fdb759-s7t8m\" (UID: \"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f\") " pod="openshift-insights/insights-operator-5885fdb759-s7t8m" Jul 29 20:25:55.995681 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.995641 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc9b1faf-5638-4fd3-ae81-d579fbf2f56f-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-s7t8m\" (UID: \"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f\") " pod="openshift-insights/insights-operator-5885fdb759-s7t8m" Jul 29 20:25:55.995681 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.995658 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66lp2\" (UniqueName: \"kubernetes.io/projected/fc9b1faf-5638-4fd3-ae81-d579fbf2f56f-kube-api-access-66lp2\") pod \"insights-operator-5885fdb759-s7t8m\" (UID: \"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f\") " pod="openshift-insights/insights-operator-5885fdb759-s7t8m" Jul 29 20:25:55.995942 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.995711 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc9b1faf-5638-4fd3-ae81-d579fbf2f56f-service-ca-bundle\") pod \"insights-operator-5885fdb759-s7t8m\" (UID: \"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f\") " pod="openshift-insights/insights-operator-5885fdb759-s7t8m" Jul 29 20:25:55.995942 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.995738 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kltv\" (UniqueName: \"kubernetes.io/projected/6a729414-a205-4b68-bee8-0c24d964a385-kube-api-access-9kltv\") pod \"cluster-monitoring-operator-548b489cb8-sgttt\" (UID: \"6a729414-a205-4b68-bee8-0c24d964a385\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt" Jul 29 20:25:55.995942 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.995792 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7fa264d5-fcc9-4972-b37e-18fc7697e367-ca-trust-extracted\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:55.995942 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:55.995828 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 20:25:55.995942 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.995832 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7fa264d5-fcc9-4972-b37e-18fc7697e367-installation-pull-secrets\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:55.995942 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:55.995849 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-555697c5b6-ljftx: secret "image-registry-tls" not found Jul 29 20:25:55.995942 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.995872 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc9b1faf-5638-4fd3-ae81-d579fbf2f56f-serving-cert\") pod \"insights-operator-5885fdb759-s7t8m\" (UID: \"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f\") " pod="openshift-insights/insights-operator-5885fdb759-s7t8m" Jul 29 20:25:55.995942 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:55.995922 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-tls podName:7fa264d5-fcc9-4972-b37e-18fc7697e367 nodeName:}" failed. No retries permitted until 2026-07-29 20:25:56.495899972 +0000 UTC m=+52.011206204 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-tls") pod "image-registry-555697c5b6-ljftx" (UID: "7fa264d5-fcc9-4972-b37e-18fc7697e367") : secret "image-registry-tls" not found Jul 29 20:25:55.996355 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.995944 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7fa264d5-fcc9-4972-b37e-18fc7697e367-image-registry-private-configuration\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:55.996355 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.996011 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a729414-a205-4b68-bee8-0c24d964a385-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sgttt\" (UID: \"6a729414-a205-4b68-bee8-0c24d964a385\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt" Jul 29 20:25:55.996355 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.996036 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7fa264d5-fcc9-4972-b37e-18fc7697e367-trusted-ca\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:55.996355 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.996077 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-bound-sa-token\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:55.996355 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.996101 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk4vw\" (UniqueName: \"kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-kube-api-access-fk4vw\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:55.996355 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.996168 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7fa264d5-fcc9-4972-b37e-18fc7697e367-ca-trust-extracted\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:55.996355 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.996301 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-certificates\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:55.996676 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.996390 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6a729414-a205-4b68-bee8-0c24d964a385-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-sgttt\" (UID: \"6a729414-a205-4b68-bee8-0c24d964a385\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt" Jul 29 20:25:55.996676 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:55.996474 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 29 20:25:55.996676 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:55.996533 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a729414-a205-4b68-bee8-0c24d964a385-cluster-monitoring-operator-tls podName:6a729414-a205-4b68-bee8-0c24d964a385 nodeName:}" failed. No retries permitted until 2026-07-29 20:25:56.496516559 +0000 UTC m=+52.011822764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6a729414-a205-4b68-bee8-0c24d964a385-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-sgttt" (UID: "6a729414-a205-4b68-bee8-0c24d964a385") : secret "cluster-monitoring-operator-tls" not found Jul 29 20:25:55.997204 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.997183 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7fa264d5-fcc9-4972-b37e-18fc7697e367-trusted-ca\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:55.999786 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.999768 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7fa264d5-fcc9-4972-b37e-18fc7697e367-installation-pull-secrets\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:55.999846 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:55.999772 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7fa264d5-fcc9-4972-b37e-18fc7697e367-image-registry-private-configuration\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:56.006323 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.006297 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk4vw\" (UniqueName: \"kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-kube-api-access-fk4vw\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:56.006510 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.006491 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-bound-sa-token\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:56.006917 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.006901 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kltv\" (UniqueName: \"kubernetes.io/projected/6a729414-a205-4b68-bee8-0c24d964a385-kube-api-access-9kltv\") pod \"cluster-monitoring-operator-548b489cb8-sgttt\" (UID: \"6a729414-a205-4b68-bee8-0c24d964a385\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt" Jul 29 20:25:56.096792 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.096756 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc9b1faf-5638-4fd3-ae81-d579fbf2f56f-serving-cert\") pod \"insights-operator-5885fdb759-s7t8m\" (UID: \"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f\") " pod="openshift-insights/insights-operator-5885fdb759-s7t8m" Jul 29 20:25:56.096965 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.096807 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f8afdfb-3427-4d4d-b0a6-1061e4571298-config\") pod \"service-ca-operator-5d6c75f84f-42vmx\" (UID: \"2f8afdfb-3427-4d4d-b0a6-1061e4571298\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-42vmx" Jul 29 20:25:56.096965 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.096919 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc9b1faf-5638-4fd3-ae81-d579fbf2f56f-tmp\") pod \"insights-operator-5885fdb759-s7t8m\" (UID: \"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f\") " pod="openshift-insights/insights-operator-5885fdb759-s7t8m" Jul 29 20:25:56.096965 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.096964 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fc9b1faf-5638-4fd3-ae81-d579fbf2f56f-snapshots\") pod \"insights-operator-5885fdb759-s7t8m\" (UID: \"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f\") " pod="openshift-insights/insights-operator-5885fdb759-s7t8m" Jul 29 20:25:56.097138 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.096982 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc9b1faf-5638-4fd3-ae81-d579fbf2f56f-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-s7t8m\" (UID: \"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f\") " pod="openshift-insights/insights-operator-5885fdb759-s7t8m" Jul 29 20:25:56.097138 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.096998 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66lp2\" (UniqueName: \"kubernetes.io/projected/fc9b1faf-5638-4fd3-ae81-d579fbf2f56f-kube-api-access-66lp2\") pod \"insights-operator-5885fdb759-s7t8m\" (UID: \"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f\") " pod="openshift-insights/insights-operator-5885fdb759-s7t8m" Jul 29 20:25:56.097138 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.097016 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz7m5\" (UniqueName: \"kubernetes.io/projected/2f8afdfb-3427-4d4d-b0a6-1061e4571298-kube-api-access-mz7m5\") pod \"service-ca-operator-5d6c75f84f-42vmx\" (UID: \"2f8afdfb-3427-4d4d-b0a6-1061e4571298\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-42vmx" Jul 29 20:25:56.097138 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.097037 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f8afdfb-3427-4d4d-b0a6-1061e4571298-serving-cert\") pod \"service-ca-operator-5d6c75f84f-42vmx\" (UID: \"2f8afdfb-3427-4d4d-b0a6-1061e4571298\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-42vmx" Jul 29 20:25:56.097138 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.097077 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc9b1faf-5638-4fd3-ae81-d579fbf2f56f-service-ca-bundle\") pod \"insights-operator-5885fdb759-s7t8m\" (UID: \"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f\") " pod="openshift-insights/insights-operator-5885fdb759-s7t8m" Jul 29 20:25:56.097703 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.097677 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc9b1faf-5638-4fd3-ae81-d579fbf2f56f-tmp\") pod \"insights-operator-5885fdb759-s7t8m\" (UID: \"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f\") " pod="openshift-insights/insights-operator-5885fdb759-s7t8m" Jul 29 20:25:56.097760 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.097688 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fc9b1faf-5638-4fd3-ae81-d579fbf2f56f-snapshots\") pod \"insights-operator-5885fdb759-s7t8m\" (UID: \"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f\") " pod="openshift-insights/insights-operator-5885fdb759-s7t8m" Jul 29 20:25:56.097807 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.097788 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc9b1faf-5638-4fd3-ae81-d579fbf2f56f-service-ca-bundle\") pod \"insights-operator-5885fdb759-s7t8m\" (UID: \"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f\") " pod="openshift-insights/insights-operator-5885fdb759-s7t8m" Jul 29 20:25:56.098226 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.098211 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc9b1faf-5638-4fd3-ae81-d579fbf2f56f-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-s7t8m\" (UID: \"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f\") " pod="openshift-insights/insights-operator-5885fdb759-s7t8m" Jul 29 20:25:56.099204 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.099188 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc9b1faf-5638-4fd3-ae81-d579fbf2f56f-serving-cert\") pod \"insights-operator-5885fdb759-s7t8m\" (UID: \"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f\") " pod="openshift-insights/insights-operator-5885fdb759-s7t8m" Jul 29 20:25:56.104954 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.104934 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66lp2\" (UniqueName: \"kubernetes.io/projected/fc9b1faf-5638-4fd3-ae81-d579fbf2f56f-kube-api-access-66lp2\") pod \"insights-operator-5885fdb759-s7t8m\" (UID: \"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f\") " pod="openshift-insights/insights-operator-5885fdb759-s7t8m" Jul 29 20:25:56.140731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.140703 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-s7t8m" Jul 29 20:25:56.197759 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.197721 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz7m5\" (UniqueName: \"kubernetes.io/projected/2f8afdfb-3427-4d4d-b0a6-1061e4571298-kube-api-access-mz7m5\") pod \"service-ca-operator-5d6c75f84f-42vmx\" (UID: \"2f8afdfb-3427-4d4d-b0a6-1061e4571298\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-42vmx" Jul 29 20:25:56.197759 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.197765 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f8afdfb-3427-4d4d-b0a6-1061e4571298-serving-cert\") pod \"service-ca-operator-5d6c75f84f-42vmx\" (UID: \"2f8afdfb-3427-4d4d-b0a6-1061e4571298\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-42vmx" Jul 29 20:25:56.197990 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.197821 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f8afdfb-3427-4d4d-b0a6-1061e4571298-config\") pod \"service-ca-operator-5d6c75f84f-42vmx\" (UID: \"2f8afdfb-3427-4d4d-b0a6-1061e4571298\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-42vmx" Jul 29 20:25:56.198346 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.198325 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f8afdfb-3427-4d4d-b0a6-1061e4571298-config\") pod \"service-ca-operator-5d6c75f84f-42vmx\" (UID: \"2f8afdfb-3427-4d4d-b0a6-1061e4571298\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-42vmx" Jul 29 20:25:56.200365 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.200341 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f8afdfb-3427-4d4d-b0a6-1061e4571298-serving-cert\") pod \"service-ca-operator-5d6c75f84f-42vmx\" (UID: \"2f8afdfb-3427-4d4d-b0a6-1061e4571298\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-42vmx" Jul 29 20:25:56.206902 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.206879 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz7m5\" (UniqueName: \"kubernetes.io/projected/2f8afdfb-3427-4d4d-b0a6-1061e4571298-kube-api-access-mz7m5\") pod \"service-ca-operator-5d6c75f84f-42vmx\" (UID: \"2f8afdfb-3427-4d4d-b0a6-1061e4571298\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-42vmx" Jul 29 20:25:56.221769 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.221745 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-42vmx" Jul 29 20:25:56.276012 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.275975 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-s7t8m"] Jul 29 20:25:56.320478 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.320451 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-s7t8m" event={"ID":"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f","Type":"ContainerStarted","Data":"c1446c223de3d6ca295aa09cdf9941aaae706b6d5aa19dbef13e2f77c5c37d25"} Jul 29 20:25:56.360607 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.360572 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-42vmx"] Jul 29 20:25:56.363426 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:25:56.363397 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f8afdfb_3427_4d4d_b0a6_1061e4571298.slice/crio-de92d1d7ac9a2987d515fe0afc9a35407470d995760e17d5a342ae66ab2b3063 WatchSource:0}: Error finding container de92d1d7ac9a2987d515fe0afc9a35407470d995760e17d5a342ae66ab2b3063: Status 404 returned error can't find the container with id de92d1d7ac9a2987d515fe0afc9a35407470d995760e17d5a342ae66ab2b3063 Jul 29 20:25:56.500321 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.500231 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a729414-a205-4b68-bee8-0c24d964a385-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sgttt\" (UID: \"6a729414-a205-4b68-bee8-0c24d964a385\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt" Jul 29 20:25:56.500321 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:56.500275 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-tls\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:56.500492 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:56.500385 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 20:25:56.500492 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:56.500396 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-555697c5b6-ljftx: secret "image-registry-tls" not found Jul 29 20:25:56.500492 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:56.500385 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 29 20:25:56.500492 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:56.500447 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-tls podName:7fa264d5-fcc9-4972-b37e-18fc7697e367 nodeName:}" failed. No retries permitted until 2026-07-29 20:25:57.500427004 +0000 UTC m=+53.015733232 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-tls") pod "image-registry-555697c5b6-ljftx" (UID: "7fa264d5-fcc9-4972-b37e-18fc7697e367") : secret "image-registry-tls" not found Jul 29 20:25:56.500492 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:56.500473 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a729414-a205-4b68-bee8-0c24d964a385-cluster-monitoring-operator-tls podName:6a729414-a205-4b68-bee8-0c24d964a385 nodeName:}" failed. No retries permitted until 2026-07-29 20:25:57.500460733 +0000 UTC m=+53.015766935 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6a729414-a205-4b68-bee8-0c24d964a385-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-sgttt" (UID: "6a729414-a205-4b68-bee8-0c24d964a385") : secret "cluster-monitoring-operator-tls" not found Jul 29 20:25:57.324164 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:57.324088 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-42vmx" event={"ID":"2f8afdfb-3427-4d4d-b0a6-1061e4571298","Type":"ContainerStarted","Data":"de92d1d7ac9a2987d515fe0afc9a35407470d995760e17d5a342ae66ab2b3063"} Jul 29 20:25:57.510127 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:57.510092 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a729414-a205-4b68-bee8-0c24d964a385-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sgttt\" (UID: \"6a729414-a205-4b68-bee8-0c24d964a385\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt" Jul 29 20:25:57.510281 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:57.510136 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-tls\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:57.510281 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:57.510243 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 29 20:25:57.510381 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:57.510314 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 20:25:57.510381 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:57.510328 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a729414-a205-4b68-bee8-0c24d964a385-cluster-monitoring-operator-tls podName:6a729414-a205-4b68-bee8-0c24d964a385 nodeName:}" failed. No retries permitted until 2026-07-29 20:25:59.510306306 +0000 UTC m=+55.025612519 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6a729414-a205-4b68-bee8-0c24d964a385-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-sgttt" (UID: "6a729414-a205-4b68-bee8-0c24d964a385") : secret "cluster-monitoring-operator-tls" not found Jul 29 20:25:57.510381 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:57.510331 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-555697c5b6-ljftx: secret "image-registry-tls" not found Jul 29 20:25:57.510381 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:57.510381 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-tls podName:7fa264d5-fcc9-4972-b37e-18fc7697e367 nodeName:}" failed. No retries permitted until 2026-07-29 20:25:59.510365703 +0000 UTC m=+55.025671910 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-tls") pod "image-registry-555697c5b6-ljftx" (UID: "7fa264d5-fcc9-4972-b37e-18fc7697e367") : secret "image-registry-tls" not found Jul 29 20:25:59.330990 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:59.330941 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-s7t8m" event={"ID":"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f","Type":"ContainerStarted","Data":"499fdd2ade8fd2219e19117ed3976308acf6c13abc0ea81f62493b639fa46d52"} Jul 29 20:25:59.332428 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:59.332403 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-42vmx" event={"ID":"2f8afdfb-3427-4d4d-b0a6-1061e4571298","Type":"ContainerStarted","Data":"900410dcc0052ec5df236268363d0510fce3f9e18b2556f83426db6e4531209c"} Jul 29 20:25:59.350197 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:59.350145 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-s7t8m" podStartSLOduration=1.510699991 podStartE2EDuration="4.350130033s" totalImagesPullingTime="2.839430042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:25:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:25:59.348890494 +0000 UTC m=+54.864196719" watchObservedRunningTime="2026-07-29 20:25:59.350130033 +0000 UTC m=+54.865436234" Jul 29 20:25:59.363995 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:59.363944 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-42vmx" podStartSLOduration=1.607718893 podStartE2EDuration="4.363926159s" totalImagesPullingTime="2.756207266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:25:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:25:59.363536045 +0000 UTC m=+54.878842297" watchObservedRunningTime="2026-07-29 20:25:59.363926159 +0000 UTC m=+54.879232382" Jul 29 20:25:59.527647 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:59.527602 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a729414-a205-4b68-bee8-0c24d964a385-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sgttt\" (UID: \"6a729414-a205-4b68-bee8-0c24d964a385\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt" Jul 29 20:25:59.527818 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:25:59.527666 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-tls\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:25:59.527818 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:59.527763 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 20:25:59.527818 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:59.527790 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-555697c5b6-ljftx: secret "image-registry-tls" not found Jul 29 20:25:59.527916 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:59.527836 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-tls podName:7fa264d5-fcc9-4972-b37e-18fc7697e367 nodeName:}" failed. No retries permitted until 2026-07-29 20:26:03.527822319 +0000 UTC m=+59.043128544 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-tls") pod "image-registry-555697c5b6-ljftx" (UID: "7fa264d5-fcc9-4972-b37e-18fc7697e367") : secret "image-registry-tls" not found Jul 29 20:25:59.527916 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:59.527763 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 29 20:25:59.527983 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:25:59.527916 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a729414-a205-4b68-bee8-0c24d964a385-cluster-monitoring-operator-tls podName:6a729414-a205-4b68-bee8-0c24d964a385 nodeName:}" failed. No retries permitted until 2026-07-29 20:26:03.527898873 +0000 UTC m=+59.043205080 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6a729414-a205-4b68-bee8-0c24d964a385-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-sgttt" (UID: "6a729414-a205-4b68-bee8-0c24d964a385") : secret "cluster-monitoring-operator-tls" not found Jul 29 20:26:00.477518 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:00.477469 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dw8kz"] Jul 29 20:26:00.479533 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:00.479511 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:00.494494 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:00.494467 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 29 20:26:00.494643 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:00.494543 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bvgmt\"" Jul 29 20:26:00.494982 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:00.494963 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 29 20:26:00.514898 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:00.514866 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dw8kz"] Jul 29 20:26:00.536365 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:00.536335 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3e7cc3ba-05fb-4d6a-9211-21196d16d214-crio-socket\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:00.536519 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:00.536371 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e7cc3ba-05fb-4d6a-9211-21196d16d214-data-volume\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:00.536519 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:00.536393 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlbr4\" (UniqueName: \"kubernetes.io/projected/3e7cc3ba-05fb-4d6a-9211-21196d16d214-kube-api-access-tlbr4\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:00.536519 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:00.536458 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e7cc3ba-05fb-4d6a-9211-21196d16d214-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:00.536635 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:00.536543 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e7cc3ba-05fb-4d6a-9211-21196d16d214-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:00.637649 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:00.637612 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3e7cc3ba-05fb-4d6a-9211-21196d16d214-crio-socket\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:00.637649 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:00.637650 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e7cc3ba-05fb-4d6a-9211-21196d16d214-data-volume\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:00.637893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:00.637673 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlbr4\" (UniqueName: \"kubernetes.io/projected/3e7cc3ba-05fb-4d6a-9211-21196d16d214-kube-api-access-tlbr4\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:00.637893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:00.637699 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e7cc3ba-05fb-4d6a-9211-21196d16d214-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:00.637893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:00.637721 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e7cc3ba-05fb-4d6a-9211-21196d16d214-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:00.637893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:00.637758 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3e7cc3ba-05fb-4d6a-9211-21196d16d214-crio-socket\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:00.637893 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:00.637811 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 20:26:00.637893 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:00.637816 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 20:26:00.637893 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:00.637884 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e7cc3ba-05fb-4d6a-9211-21196d16d214-kube-rbac-proxy-cm podName:3e7cc3ba-05fb-4d6a-9211-21196d16d214 nodeName:}" failed. No retries permitted until 2026-07-29 20:26:01.137869738 +0000 UTC m=+56.653175981 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3e7cc3ba-05fb-4d6a-9211-21196d16d214-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dw8kz" (UID: "3e7cc3ba-05fb-4d6a-9211-21196d16d214") : configmap "kube-rbac-proxy" not found Jul 29 20:26:00.637893 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:00.637898 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e7cc3ba-05fb-4d6a-9211-21196d16d214-insights-runtime-extractor-tls podName:3e7cc3ba-05fb-4d6a-9211-21196d16d214 nodeName:}" failed. No retries permitted until 2026-07-29 20:26:01.137892257 +0000 UTC m=+56.653198458 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e7cc3ba-05fb-4d6a-9211-21196d16d214-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dw8kz" (UID: "3e7cc3ba-05fb-4d6a-9211-21196d16d214") : secret "insights-runtime-extractor-tls" not found Jul 29 20:26:00.638175 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:00.637997 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e7cc3ba-05fb-4d6a-9211-21196d16d214-data-volume\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:00.646274 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:00.646247 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlbr4\" (UniqueName: \"kubernetes.io/projected/3e7cc3ba-05fb-4d6a-9211-21196d16d214-kube-api-access-tlbr4\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:01.141249 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:01.141212 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e7cc3ba-05fb-4d6a-9211-21196d16d214-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:01.141249 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:01.141255 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e7cc3ba-05fb-4d6a-9211-21196d16d214-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:01.141509 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:01.141378 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 20:26:01.141509 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:01.141389 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 20:26:01.141509 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:01.141458 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e7cc3ba-05fb-4d6a-9211-21196d16d214-insights-runtime-extractor-tls podName:3e7cc3ba-05fb-4d6a-9211-21196d16d214 nodeName:}" failed. No retries permitted until 2026-07-29 20:26:02.141436408 +0000 UTC m=+57.656742624 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e7cc3ba-05fb-4d6a-9211-21196d16d214-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dw8kz" (UID: "3e7cc3ba-05fb-4d6a-9211-21196d16d214") : secret "insights-runtime-extractor-tls" not found Jul 29 20:26:01.141509 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:01.141482 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e7cc3ba-05fb-4d6a-9211-21196d16d214-kube-rbac-proxy-cm podName:3e7cc3ba-05fb-4d6a-9211-21196d16d214 nodeName:}" failed. No retries permitted until 2026-07-29 20:26:02.141471701 +0000 UTC m=+57.656777924 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3e7cc3ba-05fb-4d6a-9211-21196d16d214-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dw8kz" (UID: "3e7cc3ba-05fb-4d6a-9211-21196d16d214") : configmap "kube-rbac-proxy" not found Jul 29 20:26:02.149010 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:02.148971 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e7cc3ba-05fb-4d6a-9211-21196d16d214-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:02.149010 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:02.149012 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e7cc3ba-05fb-4d6a-9211-21196d16d214-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:02.149552 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:02.149159 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 20:26:02.149552 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:02.149238 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e7cc3ba-05fb-4d6a-9211-21196d16d214-insights-runtime-extractor-tls podName:3e7cc3ba-05fb-4d6a-9211-21196d16d214 nodeName:}" failed. No retries permitted until 2026-07-29 20:26:04.149219462 +0000 UTC m=+59.664525682 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e7cc3ba-05fb-4d6a-9211-21196d16d214-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dw8kz" (UID: "3e7cc3ba-05fb-4d6a-9211-21196d16d214") : secret "insights-runtime-extractor-tls" not found Jul 29 20:26:02.149552 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:02.149167 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 20:26:02.149552 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:02.149301 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e7cc3ba-05fb-4d6a-9211-21196d16d214-kube-rbac-proxy-cm podName:3e7cc3ba-05fb-4d6a-9211-21196d16d214 nodeName:}" failed. No retries permitted until 2026-07-29 20:26:04.149289624 +0000 UTC m=+59.664595828 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3e7cc3ba-05fb-4d6a-9211-21196d16d214-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dw8kz" (UID: "3e7cc3ba-05fb-4d6a-9211-21196d16d214") : configmap "kube-rbac-proxy" not found Jul 29 20:26:02.229106 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:02.229073 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-d7c5v_42288536-6377-4883-90bb-69df7fc92db3/dns-node-resolver/0.log" Jul 29 20:26:03.561136 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:03.561097 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a729414-a205-4b68-bee8-0c24d964a385-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sgttt\" (UID: \"6a729414-a205-4b68-bee8-0c24d964a385\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt" Jul 29 20:26:03.561136 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:03.561137 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-tls\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:26:03.561549 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:03.561244 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 29 20:26:03.561549 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:03.561309 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a729414-a205-4b68-bee8-0c24d964a385-cluster-monitoring-operator-tls podName:6a729414-a205-4b68-bee8-0c24d964a385 nodeName:}" failed. No retries permitted until 2026-07-29 20:26:11.56129349 +0000 UTC m=+67.076599691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6a729414-a205-4b68-bee8-0c24d964a385-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-sgttt" (UID: "6a729414-a205-4b68-bee8-0c24d964a385") : secret "cluster-monitoring-operator-tls" not found Jul 29 20:26:03.561549 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:03.561246 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 20:26:03.561549 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:03.561345 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-555697c5b6-ljftx: secret "image-registry-tls" not found Jul 29 20:26:03.561549 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:03.561392 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-tls podName:7fa264d5-fcc9-4972-b37e-18fc7697e367 nodeName:}" failed. No retries permitted until 2026-07-29 20:26:11.561380195 +0000 UTC m=+67.076686401 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-tls") pod "image-registry-555697c5b6-ljftx" (UID: "7fa264d5-fcc9-4972-b37e-18fc7697e367") : secret "image-registry-tls" not found Jul 29 20:26:03.629378 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:03.629353 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zmsn6_c5ec56df-1bb7-42be-846b-17648ead013f/node-ca/0.log" Jul 29 20:26:04.166039 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:04.166001 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e7cc3ba-05fb-4d6a-9211-21196d16d214-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:04.166244 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:04.166072 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e7cc3ba-05fb-4d6a-9211-21196d16d214-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:04.166244 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:04.166153 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 20:26:04.166244 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:04.166194 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 20:26:04.166244 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:04.166235 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e7cc3ba-05fb-4d6a-9211-21196d16d214-insights-runtime-extractor-tls podName:3e7cc3ba-05fb-4d6a-9211-21196d16d214 nodeName:}" failed. No retries permitted until 2026-07-29 20:26:08.166219694 +0000 UTC m=+63.681525909 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e7cc3ba-05fb-4d6a-9211-21196d16d214-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dw8kz" (UID: "3e7cc3ba-05fb-4d6a-9211-21196d16d214") : secret "insights-runtime-extractor-tls" not found Jul 29 20:26:04.166393 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:04.166251 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e7cc3ba-05fb-4d6a-9211-21196d16d214-kube-rbac-proxy-cm podName:3e7cc3ba-05fb-4d6a-9211-21196d16d214 nodeName:}" failed. No retries permitted until 2026-07-29 20:26:08.16624403 +0000 UTC m=+63.681550232 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3e7cc3ba-05fb-4d6a-9211-21196d16d214-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dw8kz" (UID: "3e7cc3ba-05fb-4d6a-9211-21196d16d214") : configmap "kube-rbac-proxy" not found Jul 29 20:26:04.429484 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:04.429400 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qcswx_655beca7-e0da-4b4c-9001-322b63d9f11a/serve-healthcheck-canary/0.log" Jul 29 20:26:05.267189 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:05.267161 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b46zc" Jul 29 20:26:08.196263 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:08.196212 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e7cc3ba-05fb-4d6a-9211-21196d16d214-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:08.196263 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:08.196272 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e7cc3ba-05fb-4d6a-9211-21196d16d214-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:08.196807 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:08.196337 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 20:26:08.196807 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:08.196374 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 20:26:08.196807 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:08.196387 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e7cc3ba-05fb-4d6a-9211-21196d16d214-kube-rbac-proxy-cm podName:3e7cc3ba-05fb-4d6a-9211-21196d16d214 nodeName:}" failed. No retries permitted until 2026-07-29 20:26:16.196373082 +0000 UTC m=+71.711679298 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3e7cc3ba-05fb-4d6a-9211-21196d16d214-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dw8kz" (UID: "3e7cc3ba-05fb-4d6a-9211-21196d16d214") : configmap "kube-rbac-proxy" not found Jul 29 20:26:08.196807 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:08.196487 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e7cc3ba-05fb-4d6a-9211-21196d16d214-insights-runtime-extractor-tls podName:3e7cc3ba-05fb-4d6a-9211-21196d16d214 nodeName:}" failed. No retries permitted until 2026-07-29 20:26:16.196474324 +0000 UTC m=+71.711780526 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e7cc3ba-05fb-4d6a-9211-21196d16d214-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dw8kz" (UID: "3e7cc3ba-05fb-4d6a-9211-21196d16d214") : secret "insights-runtime-extractor-tls" not found Jul 29 20:26:10.717934 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:10.717893 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs\") pod \"network-metrics-daemon-vsmj9\" (UID: \"e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1\") " pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:26:10.720304 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:10.720285 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 29 20:26:10.728312 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:10.728288 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 29 20:26:10.728409 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:10.728353 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs podName:e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1 nodeName:}" failed. No retries permitted until 2026-07-29 20:27:14.728331988 +0000 UTC m=+130.243638204 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs") pod "network-metrics-daemon-vsmj9" (UID: "e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1") : secret "metrics-daemon-secret" not found Jul 29 20:26:10.819100 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:10.819062 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b24vr\" (UniqueName: \"kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr\") pod \"network-check-target-n5546\" (UID: \"ae6d959b-d6ab-4669-a6ac-0db9bafde846\") " pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:26:10.819100 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:10.819106 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret\") pod \"global-pull-secret-syncer-sv4n2\" (UID: \"b26164a7-1242-459e-8152-4a15c188d00e\") " pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:26:10.819300 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:10.819201 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a072f2ce-aee2-412d-90cf-9ecc83a109df-metrics-tls\") pod \"dns-default-nlftr\" (UID: \"a072f2ce-aee2-412d-90cf-9ecc83a109df\") " pod="openshift-dns/dns-default-nlftr" Jul 29 20:26:10.821485 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:10.821459 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a072f2ce-aee2-412d-90cf-9ecc83a109df-metrics-tls\") pod \"dns-default-nlftr\" (UID: \"a072f2ce-aee2-412d-90cf-9ecc83a109df\") " pod="openshift-dns/dns-default-nlftr" Jul 29 20:26:10.821626 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:10.821492 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 29 20:26:10.821626 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:10.821561 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 29 20:26:10.831479 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:10.831458 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b26164a7-1242-459e-8152-4a15c188d00e-original-pull-secret\") pod \"global-pull-secret-syncer-sv4n2\" (UID: \"b26164a7-1242-459e-8152-4a15c188d00e\") " pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:26:10.831754 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:10.831737 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 29 20:26:10.839746 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:10.839725 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sv4n2" Jul 29 20:26:10.841809 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:10.841782 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b24vr\" (UniqueName: \"kubernetes.io/projected/ae6d959b-d6ab-4669-a6ac-0db9bafde846-kube-api-access-b24vr\") pod \"network-check-target-n5546\" (UID: \"ae6d959b-d6ab-4669-a6ac-0db9bafde846\") " pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:26:10.962473 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:10.962438 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sv4n2"] Jul 29 20:26:10.965514 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:26:10.965482 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb26164a7_1242_459e_8152_4a15c188d00e.slice/crio-38544ed517e12879740f223ea5f97aee6b61db5906e4074038d8c427f5444d1a WatchSource:0}: Error finding container 38544ed517e12879740f223ea5f97aee6b61db5906e4074038d8c427f5444d1a: Status 404 returned error can't find the container with id 38544ed517e12879740f223ea5f97aee6b61db5906e4074038d8c427f5444d1a Jul 29 20:26:11.069811 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:11.069778 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xpqnl\"" Jul 29 20:26:11.078270 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:11.078243 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nlftr" Jul 29 20:26:11.134410 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:11.134378 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mkp42\"" Jul 29 20:26:11.142615 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:11.142593 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:26:11.208808 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:11.208774 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nlftr"] Jul 29 20:26:11.211793 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:26:11.211758 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda072f2ce_aee2_412d_90cf_9ecc83a109df.slice/crio-701ade3a4ff11bdd9fafb85ed153b0ee0865f84205aa4c308150a405a92b8215 WatchSource:0}: Error finding container 701ade3a4ff11bdd9fafb85ed153b0ee0865f84205aa4c308150a405a92b8215: Status 404 returned error can't find the container with id 701ade3a4ff11bdd9fafb85ed153b0ee0865f84205aa4c308150a405a92b8215 Jul 29 20:26:11.274637 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:11.274605 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n5546"] Jul 29 20:26:11.277866 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:26:11.277838 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae6d959b_d6ab_4669_a6ac_0db9bafde846.slice/crio-90fce6f44f467589a96ce9f490b815ebf87c68060d7199753e375a1ea76cad40 WatchSource:0}: Error finding container 90fce6f44f467589a96ce9f490b815ebf87c68060d7199753e375a1ea76cad40: Status 404 returned error can't find the container with id 90fce6f44f467589a96ce9f490b815ebf87c68060d7199753e375a1ea76cad40 Jul 29 20:26:11.364154 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:11.364114 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nlftr" event={"ID":"a072f2ce-aee2-412d-90cf-9ecc83a109df","Type":"ContainerStarted","Data":"701ade3a4ff11bdd9fafb85ed153b0ee0865f84205aa4c308150a405a92b8215"} Jul 29 20:26:11.365179 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:11.365156 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sv4n2" event={"ID":"b26164a7-1242-459e-8152-4a15c188d00e","Type":"ContainerStarted","Data":"38544ed517e12879740f223ea5f97aee6b61db5906e4074038d8c427f5444d1a"} Jul 29 20:26:11.366126 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:11.366097 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n5546" event={"ID":"ae6d959b-d6ab-4669-a6ac-0db9bafde846","Type":"ContainerStarted","Data":"90fce6f44f467589a96ce9f490b815ebf87c68060d7199753e375a1ea76cad40"} Jul 29 20:26:11.626508 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:11.626463 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a729414-a205-4b68-bee8-0c24d964a385-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sgttt\" (UID: \"6a729414-a205-4b68-bee8-0c24d964a385\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt" Jul 29 20:26:11.626698 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:11.626523 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-tls\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:26:11.626698 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:11.626638 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 29 20:26:11.626781 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:11.626726 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a729414-a205-4b68-bee8-0c24d964a385-cluster-monitoring-operator-tls podName:6a729414-a205-4b68-bee8-0c24d964a385 nodeName:}" failed. No retries permitted until 2026-07-29 20:26:27.626705331 +0000 UTC m=+83.142011557 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6a729414-a205-4b68-bee8-0c24d964a385-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-sgttt" (UID: "6a729414-a205-4b68-bee8-0c24d964a385") : secret "cluster-monitoring-operator-tls" not found Jul 29 20:26:11.629211 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:11.629190 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-tls\") pod \"image-registry-555697c5b6-ljftx\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:26:11.649474 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:11.649446 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-brdv4\"" Jul 29 20:26:11.656737 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:11.656710 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:26:11.844995 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:11.844960 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-555697c5b6-ljftx"] Jul 29 20:26:11.849229 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:26:11.849195 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fa264d5_fcc9_4972_b37e_18fc7697e367.slice/crio-5537b5a685842d2099a36c59e1fe1ca6fe16b9e9411c1efa47d7d38085b795b2 WatchSource:0}: Error finding container 5537b5a685842d2099a36c59e1fe1ca6fe16b9e9411c1efa47d7d38085b795b2: Status 404 returned error can't find the container with id 5537b5a685842d2099a36c59e1fe1ca6fe16b9e9411c1efa47d7d38085b795b2 Jul 29 20:26:12.372718 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:12.372621 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-555697c5b6-ljftx" event={"ID":"7fa264d5-fcc9-4972-b37e-18fc7697e367","Type":"ContainerStarted","Data":"feae1006f76e76822edd11d4510f00058a514ea4752a3c7a7ea65db6250fd4ce"} Jul 29 20:26:12.372718 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:12.372682 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-555697c5b6-ljftx" event={"ID":"7fa264d5-fcc9-4972-b37e-18fc7697e367","Type":"ContainerStarted","Data":"5537b5a685842d2099a36c59e1fe1ca6fe16b9e9411c1efa47d7d38085b795b2"} Jul 29 20:26:12.373110 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:12.373090 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:26:12.393354 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:12.393270 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-555697c5b6-ljftx" podStartSLOduration=17.393249317 podStartE2EDuration="17.393249317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:25:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:26:12.393099788 +0000 UTC m=+67.908406040" watchObservedRunningTime="2026-07-29 20:26:12.393249317 +0000 UTC m=+67.908555541" Jul 29 20:26:16.269347 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:16.269300 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e7cc3ba-05fb-4d6a-9211-21196d16d214-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:16.269818 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:16.269360 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e7cc3ba-05fb-4d6a-9211-21196d16d214-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:16.269818 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:16.269453 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 20:26:16.269818 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:16.269546 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e7cc3ba-05fb-4d6a-9211-21196d16d214-insights-runtime-extractor-tls podName:3e7cc3ba-05fb-4d6a-9211-21196d16d214 nodeName:}" failed. No retries permitted until 2026-07-29 20:26:32.269522419 +0000 UTC m=+87.784828645 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e7cc3ba-05fb-4d6a-9211-21196d16d214-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dw8kz" (UID: "3e7cc3ba-05fb-4d6a-9211-21196d16d214") : secret "insights-runtime-extractor-tls" not found Jul 29 20:26:16.270013 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:16.269960 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e7cc3ba-05fb-4d6a-9211-21196d16d214-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:17.390239 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:17.390191 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sv4n2" event={"ID":"b26164a7-1242-459e-8152-4a15c188d00e","Type":"ContainerStarted","Data":"cd0237b7796c904a79b0cc0a346778555d2da9acac450ae10ac101c64bc65016"} Jul 29 20:26:17.391818 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:17.391793 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n5546" event={"ID":"ae6d959b-d6ab-4669-a6ac-0db9bafde846","Type":"ContainerStarted","Data":"2db8addd6fc1545e985e89e50a4a8ef8b1bbbba3afd120b805b040b6f373e7b5"} Jul 29 20:26:17.391933 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:17.391866 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:26:17.393515 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:17.393487 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nlftr" event={"ID":"a072f2ce-aee2-412d-90cf-9ecc83a109df","Type":"ContainerStarted","Data":"9b577dbd8e1906d95794e9f78de6382bd33f3971ea2d4e0cded2b94a88f98e4d"} Jul 29 20:26:17.393644 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:17.393531 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nlftr" event={"ID":"a072f2ce-aee2-412d-90cf-9ecc83a109df","Type":"ContainerStarted","Data":"40afa3c6d3ad4e9c50b0106bd482a49298b08f674cf1f3edab06424071511f5f"} Jul 29 20:26:17.393712 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:17.393657 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nlftr" Jul 29 20:26:17.406076 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:17.406007 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-sv4n2" podStartSLOduration=66.973346681 podStartE2EDuration="1m12.405989296s" totalImagesPullingTime="5.432642615s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:25:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:26:17.405180901 +0000 UTC m=+72.920487135" watchObservedRunningTime="2026-07-29 20:26:17.405989296 +0000 UTC m=+72.921295520" Jul 29 20:26:17.421777 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:17.421712 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-n5546" podStartSLOduration=67.31051488 podStartE2EDuration="1m12.421695392s" totalImagesPullingTime="5.111180512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:25:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:26:17.421033029 +0000 UTC m=+72.936339253" watchObservedRunningTime="2026-07-29 20:26:17.421695392 +0000 UTC m=+72.937001617" Jul 29 20:26:17.437178 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:17.437114 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nlftr" podStartSLOduration=34.267551051 podStartE2EDuration="39.437097187s" totalImagesPullingTime="5.169546136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:25:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:26:17.436085678 +0000 UTC m=+72.951391901" watchObservedRunningTime="2026-07-29 20:26:17.437097187 +0000 UTC m=+72.952403411" Jul 29 20:26:21.758712 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.758675 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-555697c5b6-ljftx"] Jul 29 20:26:21.781893 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.781861 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7548986db6-tz6ww"] Jul 29 20:26:21.784485 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.784470 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7548986db6-tz6ww" Jul 29 20:26:21.787542 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.787523 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 29 20:26:21.787828 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.787815 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 29 20:26:21.788289 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.788274 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 29 20:26:21.788466 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.788455 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 29 20:26:21.800569 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.800543 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7548986db6-tz6ww"] Jul 29 20:26:21.806859 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.806813 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-647c86dff8-ls4m7"] Jul 29 20:26:21.808607 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.808585 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:21.825749 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.825719 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-647c86dff8-ls4m7"] Jul 29 20:26:21.913291 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.913249 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ceddaa08-0819-4908-bf58-03b8798c1a84-installation-pull-secrets\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:21.913291 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.913291 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k9mg\" (UniqueName: \"kubernetes.io/projected/ceddaa08-0819-4908-bf58-03b8798c1a84-kube-api-access-2k9mg\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:21.913540 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.913315 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5m9s\" (UniqueName: \"kubernetes.io/projected/282d9a31-a372-49d3-989a-ca747f224d25-kube-api-access-g5m9s\") pod \"klusterlet-addon-workmgr-7548986db6-tz6ww\" (UID: \"282d9a31-a372-49d3-989a-ca747f224d25\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7548986db6-tz6ww" Jul 29 20:26:21.913540 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.913366 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ceddaa08-0819-4908-bf58-03b8798c1a84-bound-sa-token\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:21.913540 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.913403 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/282d9a31-a372-49d3-989a-ca747f224d25-klusterlet-config\") pod \"klusterlet-addon-workmgr-7548986db6-tz6ww\" (UID: \"282d9a31-a372-49d3-989a-ca747f224d25\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7548986db6-tz6ww" Jul 29 20:26:21.913540 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.913441 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ceddaa08-0819-4908-bf58-03b8798c1a84-registry-tls\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:21.913540 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.913465 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ceddaa08-0819-4908-bf58-03b8798c1a84-registry-certificates\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:21.913540 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.913530 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ceddaa08-0819-4908-bf58-03b8798c1a84-image-registry-private-configuration\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:21.913805 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.913586 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ceddaa08-0819-4908-bf58-03b8798c1a84-ca-trust-extracted\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:21.913805 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.913618 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ceddaa08-0819-4908-bf58-03b8798c1a84-trusted-ca\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:21.913805 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:21.913657 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/282d9a31-a372-49d3-989a-ca747f224d25-tmp\") pod \"klusterlet-addon-workmgr-7548986db6-tz6ww\" (UID: \"282d9a31-a372-49d3-989a-ca747f224d25\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7548986db6-tz6ww" Jul 29 20:26:22.014961 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.014866 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ceddaa08-0819-4908-bf58-03b8798c1a84-image-registry-private-configuration\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:22.014961 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.014926 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ceddaa08-0819-4908-bf58-03b8798c1a84-ca-trust-extracted\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:22.014961 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.014956 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ceddaa08-0819-4908-bf58-03b8798c1a84-trusted-ca\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:22.015270 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.014989 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/282d9a31-a372-49d3-989a-ca747f224d25-tmp\") pod \"klusterlet-addon-workmgr-7548986db6-tz6ww\" (UID: \"282d9a31-a372-49d3-989a-ca747f224d25\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7548986db6-tz6ww" Jul 29 20:26:22.015270 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.015040 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ceddaa08-0819-4908-bf58-03b8798c1a84-installation-pull-secrets\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:22.015270 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.015080 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k9mg\" (UniqueName: \"kubernetes.io/projected/ceddaa08-0819-4908-bf58-03b8798c1a84-kube-api-access-2k9mg\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:22.015270 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.015104 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5m9s\" (UniqueName: \"kubernetes.io/projected/282d9a31-a372-49d3-989a-ca747f224d25-kube-api-access-g5m9s\") pod \"klusterlet-addon-workmgr-7548986db6-tz6ww\" (UID: \"282d9a31-a372-49d3-989a-ca747f224d25\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7548986db6-tz6ww" Jul 29 20:26:22.015270 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.015128 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ceddaa08-0819-4908-bf58-03b8798c1a84-bound-sa-token\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:22.015270 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.015152 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/282d9a31-a372-49d3-989a-ca747f224d25-klusterlet-config\") pod \"klusterlet-addon-workmgr-7548986db6-tz6ww\" (UID: \"282d9a31-a372-49d3-989a-ca747f224d25\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7548986db6-tz6ww" Jul 29 20:26:22.015270 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.015177 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ceddaa08-0819-4908-bf58-03b8798c1a84-registry-tls\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:22.015270 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.015202 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ceddaa08-0819-4908-bf58-03b8798c1a84-registry-certificates\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:22.015636 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.015401 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/282d9a31-a372-49d3-989a-ca747f224d25-tmp\") pod \"klusterlet-addon-workmgr-7548986db6-tz6ww\" (UID: \"282d9a31-a372-49d3-989a-ca747f224d25\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7548986db6-tz6ww" Jul 29 20:26:22.015636 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.015423 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ceddaa08-0819-4908-bf58-03b8798c1a84-ca-trust-extracted\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:22.016253 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.016226 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ceddaa08-0819-4908-bf58-03b8798c1a84-registry-certificates\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:22.016471 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.016450 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ceddaa08-0819-4908-bf58-03b8798c1a84-trusted-ca\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:22.017551 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.017525 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ceddaa08-0819-4908-bf58-03b8798c1a84-image-registry-private-configuration\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:22.017691 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.017674 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ceddaa08-0819-4908-bf58-03b8798c1a84-installation-pull-secrets\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:22.017803 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.017779 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/282d9a31-a372-49d3-989a-ca747f224d25-klusterlet-config\") pod \"klusterlet-addon-workmgr-7548986db6-tz6ww\" (UID: \"282d9a31-a372-49d3-989a-ca747f224d25\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7548986db6-tz6ww" Jul 29 20:26:22.017913 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.017898 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ceddaa08-0819-4908-bf58-03b8798c1a84-registry-tls\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:22.026611 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.026587 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ceddaa08-0819-4908-bf58-03b8798c1a84-bound-sa-token\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:22.028400 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.028375 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5m9s\" (UniqueName: \"kubernetes.io/projected/282d9a31-a372-49d3-989a-ca747f224d25-kube-api-access-g5m9s\") pod \"klusterlet-addon-workmgr-7548986db6-tz6ww\" (UID: \"282d9a31-a372-49d3-989a-ca747f224d25\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7548986db6-tz6ww" Jul 29 20:26:22.028852 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.028836 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k9mg\" (UniqueName: \"kubernetes.io/projected/ceddaa08-0819-4908-bf58-03b8798c1a84-kube-api-access-2k9mg\") pod \"image-registry-647c86dff8-ls4m7\" (UID: \"ceddaa08-0819-4908-bf58-03b8798c1a84\") " pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:22.094095 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.094041 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7548986db6-tz6ww" Jul 29 20:26:22.116818 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.116792 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:22.233710 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.233678 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7548986db6-tz6ww"] Jul 29 20:26:22.236786 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:26:22.236761 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod282d9a31_a372_49d3_989a_ca747f224d25.slice/crio-4d43a26a76b75b7c85190b5acd6217b3e61e90a933ac6c641ed810e62ba037f7 WatchSource:0}: Error finding container 4d43a26a76b75b7c85190b5acd6217b3e61e90a933ac6c641ed810e62ba037f7: Status 404 returned error can't find the container with id 4d43a26a76b75b7c85190b5acd6217b3e61e90a933ac6c641ed810e62ba037f7 Jul 29 20:26:22.254939 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.254904 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-647c86dff8-ls4m7"] Jul 29 20:26:22.258826 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:26:22.258798 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceddaa08_0819_4908_bf58_03b8798c1a84.slice/crio-8773f6dda51d99b01d3f87701e8e8dde867d3ea1f646a705c3cca7877bba71a0 WatchSource:0}: Error finding container 8773f6dda51d99b01d3f87701e8e8dde867d3ea1f646a705c3cca7877bba71a0: Status 404 returned error can't find the container with id 8773f6dda51d99b01d3f87701e8e8dde867d3ea1f646a705c3cca7877bba71a0 Jul 29 20:26:22.412195 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.412155 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" event={"ID":"ceddaa08-0819-4908-bf58-03b8798c1a84","Type":"ContainerStarted","Data":"d5eceec476a8ec0eca8f5a250153c70a662232162c5c1bebbc3ad34ed5e116d5"} Jul 29 20:26:22.412393 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.412213 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" event={"ID":"ceddaa08-0819-4908-bf58-03b8798c1a84","Type":"ContainerStarted","Data":"8773f6dda51d99b01d3f87701e8e8dde867d3ea1f646a705c3cca7877bba71a0"} Jul 29 20:26:22.412393 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.412237 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:22.413367 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:22.413340 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7548986db6-tz6ww" event={"ID":"282d9a31-a372-49d3-989a-ca747f224d25","Type":"ContainerStarted","Data":"4d43a26a76b75b7c85190b5acd6217b3e61e90a933ac6c641ed810e62ba037f7"} Jul 29 20:26:27.400268 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:27.400163 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nlftr" Jul 29 20:26:27.420501 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:27.420448 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" podStartSLOduration=6.420426571 podStartE2EDuration="6.420426571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:26:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:26:22.438597112 +0000 UTC m=+77.953903335" watchObservedRunningTime="2026-07-29 20:26:27.420426571 +0000 UTC m=+82.935732795" Jul 29 20:26:27.437888 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:27.437854 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7548986db6-tz6ww" event={"ID":"282d9a31-a372-49d3-989a-ca747f224d25","Type":"ContainerStarted","Data":"08e637a747f8e7ca00251b66e352d58d381122416e3140f025594572f3976831"} Jul 29 20:26:27.438092 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:27.438070 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7548986db6-tz6ww" Jul 29 20:26:27.439668 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:27.439646 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7548986db6-tz6ww" Jul 29 20:26:27.454881 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:27.454827 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7548986db6-tz6ww" podStartSLOduration=2.317596483 podStartE2EDuration="6.454811261s" totalImagesPullingTime="4.137214778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:26:27.453517606 +0000 UTC m=+82.968823831" watchObservedRunningTime="2026-07-29 20:26:27.454811261 +0000 UTC m=+82.970117484" Jul 29 20:26:27.666299 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:27.666188 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a729414-a205-4b68-bee8-0c24d964a385-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sgttt\" (UID: \"6a729414-a205-4b68-bee8-0c24d964a385\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt" Jul 29 20:26:27.668547 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:27.668530 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6a729414-a205-4b68-bee8-0c24d964a385-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sgttt\" (UID: \"6a729414-a205-4b68-bee8-0c24d964a385\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt" Jul 29 20:26:27.841420 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:27.841387 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9qlzx\"" Jul 29 20:26:27.848416 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:27.848395 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt" Jul 29 20:26:27.975377 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:27.975319 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt"] Jul 29 20:26:27.978181 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:26:27.978155 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a729414_a205_4b68_bee8_0c24d964a385.slice/crio-a8500a8b31230384aac734479e62903b3670c03f3a09290e295a237f11efb7e3 WatchSource:0}: Error finding container a8500a8b31230384aac734479e62903b3670c03f3a09290e295a237f11efb7e3: Status 404 returned error can't find the container with id a8500a8b31230384aac734479e62903b3670c03f3a09290e295a237f11efb7e3 Jul 29 20:26:28.442287 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:28.442247 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt" event={"ID":"6a729414-a205-4b68-bee8-0c24d964a385","Type":"ContainerStarted","Data":"a8500a8b31230384aac734479e62903b3670c03f3a09290e295a237f11efb7e3"} Jul 29 20:26:30.450486 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:30.450444 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt" event={"ID":"6a729414-a205-4b68-bee8-0c24d964a385","Type":"ContainerStarted","Data":"c4583797afb0f23da92a3d95a0a4f0d8ee70a61cb908e9ff702c6511534e5381"} Jul 29 20:26:30.468434 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:30.468383 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sgttt" podStartSLOduration=33.6293877 podStartE2EDuration="35.468364866s" totalImagesPullingTime="1.838977166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:25:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:26:30.467694486 +0000 UTC m=+85.983000710" watchObservedRunningTime="2026-07-29 20:26:30.468364866 +0000 UTC m=+85.983671081" Jul 29 20:26:31.763063 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:31.763019 2559 patch_prober.go:28] interesting pod/image-registry-555697c5b6-ljftx container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 29 20:26:31.763451 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:31.763109 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-555697c5b6-ljftx" podUID="7fa264d5-fcc9-4972-b37e-18fc7697e367" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 20:26:32.301678 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:32.301634 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e7cc3ba-05fb-4d6a-9211-21196d16d214-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:32.303980 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:32.303957 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e7cc3ba-05fb-4d6a-9211-21196d16d214-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dw8kz\" (UID: \"3e7cc3ba-05fb-4d6a-9211-21196d16d214\") " pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:32.591357 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:32.591271 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bvgmt\"" Jul 29 20:26:32.599181 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:32.599161 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dw8kz" Jul 29 20:26:32.749744 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:32.749714 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dw8kz"] Jul 29 20:26:32.753655 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:26:32.753626 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e7cc3ba_05fb_4d6a_9211_21196d16d214.slice/crio-95de6301022eeb352d956b1ebb241279910a84701865842f7d1d01081a942a86 WatchSource:0}: Error finding container 95de6301022eeb352d956b1ebb241279910a84701865842f7d1d01081a942a86: Status 404 returned error can't find the container with id 95de6301022eeb352d956b1ebb241279910a84701865842f7d1d01081a942a86 Jul 29 20:26:33.418592 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:33.417323 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-zvmj7"] Jul 29 20:26:33.420859 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:33.420831 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-zvmj7" Jul 29 20:26:33.424671 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:33.424647 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 29 20:26:33.424883 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:33.424868 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 29 20:26:33.425100 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:33.425084 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-twt2f\"" Jul 29 20:26:33.425321 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:33.425306 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 29 20:26:33.430519 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:33.430458 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-zvmj7"] Jul 29 20:26:33.463296 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:33.463263 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dw8kz" event={"ID":"3e7cc3ba-05fb-4d6a-9211-21196d16d214","Type":"ContainerStarted","Data":"95de6301022eeb352d956b1ebb241279910a84701865842f7d1d01081a942a86"} Jul 29 20:26:33.510885 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:33.510735 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2288f7e-999c-4428-8e45-8e05672a6e78-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-zvmj7\" (UID: \"b2288f7e-999c-4428-8e45-8e05672a6e78\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-zvmj7" Jul 29 20:26:33.510885 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:33.510782 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b2288f7e-999c-4428-8e45-8e05672a6e78-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-zvmj7\" (UID: \"b2288f7e-999c-4428-8e45-8e05672a6e78\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-zvmj7" Jul 29 20:26:33.510885 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:33.510815 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz8lq\" (UniqueName: \"kubernetes.io/projected/b2288f7e-999c-4428-8e45-8e05672a6e78-kube-api-access-wz8lq\") pod \"prometheus-operator-84d69f794d-zvmj7\" (UID: \"b2288f7e-999c-4428-8e45-8e05672a6e78\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-zvmj7" Jul 29 20:26:33.510885 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:33.510856 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b2288f7e-999c-4428-8e45-8e05672a6e78-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-zvmj7\" (UID: \"b2288f7e-999c-4428-8e45-8e05672a6e78\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-zvmj7" Jul 29 20:26:33.611540 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:33.611463 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2288f7e-999c-4428-8e45-8e05672a6e78-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-zvmj7\" (UID: \"b2288f7e-999c-4428-8e45-8e05672a6e78\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-zvmj7" Jul 29 20:26:33.611540 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:33.611511 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b2288f7e-999c-4428-8e45-8e05672a6e78-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-zvmj7\" (UID: \"b2288f7e-999c-4428-8e45-8e05672a6e78\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-zvmj7" Jul 29 20:26:33.611859 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:33.611625 2559 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 29 20:26:33.611859 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:33.611638 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz8lq\" (UniqueName: \"kubernetes.io/projected/b2288f7e-999c-4428-8e45-8e05672a6e78-kube-api-access-wz8lq\") pod \"prometheus-operator-84d69f794d-zvmj7\" (UID: \"b2288f7e-999c-4428-8e45-8e05672a6e78\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-zvmj7" Jul 29 20:26:33.611859 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:33.611689 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2288f7e-999c-4428-8e45-8e05672a6e78-prometheus-operator-tls podName:b2288f7e-999c-4428-8e45-8e05672a6e78 nodeName:}" failed. No retries permitted until 2026-07-29 20:26:34.111670116 +0000 UTC m=+89.626976334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/b2288f7e-999c-4428-8e45-8e05672a6e78-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-zvmj7" (UID: "b2288f7e-999c-4428-8e45-8e05672a6e78") : secret "prometheus-operator-tls" not found Jul 29 20:26:33.611859 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:33.611707 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b2288f7e-999c-4428-8e45-8e05672a6e78-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-zvmj7\" (UID: \"b2288f7e-999c-4428-8e45-8e05672a6e78\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-zvmj7" Jul 29 20:26:33.612361 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:33.612328 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2288f7e-999c-4428-8e45-8e05672a6e78-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-zvmj7\" (UID: \"b2288f7e-999c-4428-8e45-8e05672a6e78\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-zvmj7" Jul 29 20:26:33.614914 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:33.614887 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b2288f7e-999c-4428-8e45-8e05672a6e78-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-zvmj7\" (UID: \"b2288f7e-999c-4428-8e45-8e05672a6e78\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-zvmj7" Jul 29 20:26:33.621590 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:33.621566 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz8lq\" (UniqueName: \"kubernetes.io/projected/b2288f7e-999c-4428-8e45-8e05672a6e78-kube-api-access-wz8lq\") pod \"prometheus-operator-84d69f794d-zvmj7\" (UID: \"b2288f7e-999c-4428-8e45-8e05672a6e78\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-zvmj7" Jul 29 20:26:34.115653 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:34.115610 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b2288f7e-999c-4428-8e45-8e05672a6e78-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-zvmj7\" (UID: \"b2288f7e-999c-4428-8e45-8e05672a6e78\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-zvmj7" Jul 29 20:26:34.118385 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:34.118356 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b2288f7e-999c-4428-8e45-8e05672a6e78-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-zvmj7\" (UID: \"b2288f7e-999c-4428-8e45-8e05672a6e78\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-zvmj7" Jul 29 20:26:34.335976 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:34.335943 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-zvmj7" Jul 29 20:26:34.484270 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:34.484236 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-zvmj7"] Jul 29 20:26:34.488204 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:26:34.488179 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2288f7e_999c_4428_8e45_8e05672a6e78.slice/crio-764a738030e7d5fcb9e41b0d4b5a3a2663361a1849881163528a42162b0d685f WatchSource:0}: Error finding container 764a738030e7d5fcb9e41b0d4b5a3a2663361a1849881163528a42162b0d685f: Status 404 returned error can't find the container with id 764a738030e7d5fcb9e41b0d4b5a3a2663361a1849881163528a42162b0d685f Jul 29 20:26:35.471963 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:35.471929 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-zvmj7" event={"ID":"b2288f7e-999c-4428-8e45-8e05672a6e78","Type":"ContainerStarted","Data":"764a738030e7d5fcb9e41b0d4b5a3a2663361a1849881163528a42162b0d685f"} Jul 29 20:26:38.485201 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:38.485159 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-zvmj7" event={"ID":"b2288f7e-999c-4428-8e45-8e05672a6e78","Type":"ContainerStarted","Data":"2ecc6842195af3c9dc1bf8544411087bdc8bbc592b0836b1f091b89a9c6a42ec"} Jul 29 20:26:38.485680 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:38.485248 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-zvmj7" event={"ID":"b2288f7e-999c-4428-8e45-8e05672a6e78","Type":"ContainerStarted","Data":"a460f8d42c6b1d5a6e92d02bdafd2de0ad528f52869b59a8dc63d75f10c79429"} Jul 29 20:26:38.487083 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:38.487038 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dw8kz" event={"ID":"3e7cc3ba-05fb-4d6a-9211-21196d16d214","Type":"ContainerStarted","Data":"a0936ebc1a4255ce5178a9301d3690d481762c5f98a592c616a10b5d4c33dfba"} Jul 29 20:26:38.503471 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:38.503417 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-zvmj7" podStartSLOduration=2.028110107 podStartE2EDuration="5.503399276s" totalImagesPullingTime="3.475289169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:26:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:26:38.502221105 +0000 UTC m=+94.017527329" watchObservedRunningTime="2026-07-29 20:26:38.503399276 +0000 UTC m=+94.018705496" Jul 29 20:26:40.814835 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.814801 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7"] Jul 29 20:26:40.820679 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.820658 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7" Jul 29 20:26:40.823578 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.823556 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 29 20:26:40.823766 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.823748 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4hcl5\"" Jul 29 20:26:40.824726 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.824706 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 29 20:26:40.846286 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.846264 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7"] Jul 29 20:26:40.847779 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.847756 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-f5blf"] Jul 29 20:26:40.854615 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.854593 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:40.857351 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.857328 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 29 20:26:40.857696 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.857674 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 29 20:26:40.858193 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.857751 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 29 20:26:40.858193 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.858070 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-t564f\"" Jul 29 20:26:40.977368 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.977328 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5797ad79-9ff2-4480-b284-ad4ce5000915-sys\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:40.977368 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.977371 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb8q9\" (UniqueName: \"kubernetes.io/projected/936bcf63-8ee2-461b-8e1e-783a3c0ac208-kube-api-access-sb8q9\") pod \"openshift-state-metrics-b8f49f5c6-5bcs7\" (UID: \"936bcf63-8ee2-461b-8e1e-783a3c0ac208\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7" Jul 29 20:26:40.977613 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.977400 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/936bcf63-8ee2-461b-8e1e-783a3c0ac208-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-5bcs7\" (UID: \"936bcf63-8ee2-461b-8e1e-783a3c0ac208\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7" Jul 29 20:26:40.977613 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.977523 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5797ad79-9ff2-4480-b284-ad4ce5000915-node-exporter-textfile\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:40.977613 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.977559 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5797ad79-9ff2-4480-b284-ad4ce5000915-node-exporter-accelerators-collector-config\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:40.977613 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.977587 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8899d\" (UniqueName: \"kubernetes.io/projected/5797ad79-9ff2-4480-b284-ad4ce5000915-kube-api-access-8899d\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:40.977807 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.977645 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5797ad79-9ff2-4480-b284-ad4ce5000915-node-exporter-tls\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:40.977807 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.977699 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/936bcf63-8ee2-461b-8e1e-783a3c0ac208-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-5bcs7\" (UID: \"936bcf63-8ee2-461b-8e1e-783a3c0ac208\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7" Jul 29 20:26:40.977807 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.977731 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5797ad79-9ff2-4480-b284-ad4ce5000915-node-exporter-wtmp\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:40.977807 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.977758 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5797ad79-9ff2-4480-b284-ad4ce5000915-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:40.977807 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.977787 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/936bcf63-8ee2-461b-8e1e-783a3c0ac208-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-5bcs7\" (UID: \"936bcf63-8ee2-461b-8e1e-783a3c0ac208\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7" Jul 29 20:26:40.978185 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.978164 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5797ad79-9ff2-4480-b284-ad4ce5000915-root\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:40.978392 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:40.978195 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5797ad79-9ff2-4480-b284-ad4ce5000915-metrics-client-ca\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:41.079600 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.079501 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5797ad79-9ff2-4480-b284-ad4ce5000915-sys\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:41.079600 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.079553 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb8q9\" (UniqueName: \"kubernetes.io/projected/936bcf63-8ee2-461b-8e1e-783a3c0ac208-kube-api-access-sb8q9\") pod \"openshift-state-metrics-b8f49f5c6-5bcs7\" (UID: \"936bcf63-8ee2-461b-8e1e-783a3c0ac208\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7" Jul 29 20:26:41.079851 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.079634 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5797ad79-9ff2-4480-b284-ad4ce5000915-sys\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:41.079851 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.079699 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/936bcf63-8ee2-461b-8e1e-783a3c0ac208-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-5bcs7\" (UID: \"936bcf63-8ee2-461b-8e1e-783a3c0ac208\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7" Jul 29 20:26:41.079851 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.079770 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5797ad79-9ff2-4480-b284-ad4ce5000915-node-exporter-textfile\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:41.079851 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.079802 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5797ad79-9ff2-4480-b284-ad4ce5000915-node-exporter-accelerators-collector-config\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:41.080032 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:41.079850 2559 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 29 20:26:41.080032 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:26:41.079924 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/936bcf63-8ee2-461b-8e1e-783a3c0ac208-openshift-state-metrics-tls podName:936bcf63-8ee2-461b-8e1e-783a3c0ac208 nodeName:}" failed. No retries permitted until 2026-07-29 20:26:41.579902233 +0000 UTC m=+97.095208455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/936bcf63-8ee2-461b-8e1e-783a3c0ac208-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-5bcs7" (UID: "936bcf63-8ee2-461b-8e1e-783a3c0ac208") : secret "openshift-state-metrics-tls" not found Jul 29 20:26:41.080032 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.079922 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8899d\" (UniqueName: \"kubernetes.io/projected/5797ad79-9ff2-4480-b284-ad4ce5000915-kube-api-access-8899d\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:41.080032 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.080001 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5797ad79-9ff2-4480-b284-ad4ce5000915-node-exporter-tls\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:41.080250 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.080030 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/936bcf63-8ee2-461b-8e1e-783a3c0ac208-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-5bcs7\" (UID: \"936bcf63-8ee2-461b-8e1e-783a3c0ac208\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7" Jul 29 20:26:41.080250 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.080084 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5797ad79-9ff2-4480-b284-ad4ce5000915-node-exporter-wtmp\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:41.080250 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.080115 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5797ad79-9ff2-4480-b284-ad4ce5000915-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:41.080250 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.080148 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/936bcf63-8ee2-461b-8e1e-783a3c0ac208-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-5bcs7\" (UID: \"936bcf63-8ee2-461b-8e1e-783a3c0ac208\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7" Jul 29 20:26:41.080250 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.080196 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5797ad79-9ff2-4480-b284-ad4ce5000915-root\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:41.080250 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.080221 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5797ad79-9ff2-4480-b284-ad4ce5000915-metrics-client-ca\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:41.080527 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.080264 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5797ad79-9ff2-4480-b284-ad4ce5000915-node-exporter-wtmp\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:41.080527 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.080399 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5797ad79-9ff2-4480-b284-ad4ce5000915-node-exporter-accelerators-collector-config\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:41.080527 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.080456 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5797ad79-9ff2-4480-b284-ad4ce5000915-root\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:41.080527 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.080506 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5797ad79-9ff2-4480-b284-ad4ce5000915-node-exporter-textfile\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:41.080763 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.080747 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5797ad79-9ff2-4480-b284-ad4ce5000915-metrics-client-ca\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:41.081209 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.081185 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/936bcf63-8ee2-461b-8e1e-783a3c0ac208-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-5bcs7\" (UID: \"936bcf63-8ee2-461b-8e1e-783a3c0ac208\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7" Jul 29 20:26:41.082746 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.082710 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5797ad79-9ff2-4480-b284-ad4ce5000915-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:41.082850 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.082795 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5797ad79-9ff2-4480-b284-ad4ce5000915-node-exporter-tls\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:41.083580 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.083554 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/936bcf63-8ee2-461b-8e1e-783a3c0ac208-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-5bcs7\" (UID: \"936bcf63-8ee2-461b-8e1e-783a3c0ac208\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7" Jul 29 20:26:41.090020 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.089998 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb8q9\" (UniqueName: \"kubernetes.io/projected/936bcf63-8ee2-461b-8e1e-783a3c0ac208-kube-api-access-sb8q9\") pod \"openshift-state-metrics-b8f49f5c6-5bcs7\" (UID: \"936bcf63-8ee2-461b-8e1e-783a3c0ac208\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7" Jul 29 20:26:41.091748 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.091725 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8899d\" (UniqueName: \"kubernetes.io/projected/5797ad79-9ff2-4480-b284-ad4ce5000915-kube-api-access-8899d\") pod \"node-exporter-f5blf\" (UID: \"5797ad79-9ff2-4480-b284-ad4ce5000915\") " pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:41.167073 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.167032 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f5blf" Jul 29 20:26:41.390137 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:26:41.390109 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5797ad79_9ff2_4480_b284_ad4ce5000915.slice/crio-d4028652d911f037bfcb5984e46dd72205b35d137e7b3cca07ba708659a0efc7 WatchSource:0}: Error finding container d4028652d911f037bfcb5984e46dd72205b35d137e7b3cca07ba708659a0efc7: Status 404 returned error can't find the container with id d4028652d911f037bfcb5984e46dd72205b35d137e7b3cca07ba708659a0efc7 Jul 29 20:26:41.499585 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.499558 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f5blf" event={"ID":"5797ad79-9ff2-4480-b284-ad4ce5000915","Type":"ContainerStarted","Data":"d4028652d911f037bfcb5984e46dd72205b35d137e7b3cca07ba708659a0efc7"} Jul 29 20:26:41.501263 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.501230 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dw8kz" event={"ID":"3e7cc3ba-05fb-4d6a-9211-21196d16d214","Type":"ContainerStarted","Data":"84076ed7dc25184402779e11798c88b150b105ccbf37e5f59511922484d925a8"} Jul 29 20:26:41.584753 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.584723 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/936bcf63-8ee2-461b-8e1e-783a3c0ac208-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-5bcs7\" (UID: \"936bcf63-8ee2-461b-8e1e-783a3c0ac208\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7" Jul 29 20:26:41.586919 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.586897 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/936bcf63-8ee2-461b-8e1e-783a3c0ac208-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-5bcs7\" (UID: \"936bcf63-8ee2-461b-8e1e-783a3c0ac208\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7" Jul 29 20:26:41.731997 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.731927 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7" Jul 29 20:26:41.763338 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.763311 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:26:41.875154 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:41.875125 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7"] Jul 29 20:26:41.878478 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:26:41.878447 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod936bcf63_8ee2_461b_8e1e_783a3c0ac208.slice/crio-9c39eae7859b05f0735f57fb272b541931a2d01a1878e7507e6077bab7633ad9 WatchSource:0}: Error finding container 9c39eae7859b05f0735f57fb272b541931a2d01a1878e7507e6077bab7633ad9: Status 404 returned error can't find the container with id 9c39eae7859b05f0735f57fb272b541931a2d01a1878e7507e6077bab7633ad9 Jul 29 20:26:42.508886 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:42.508851 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7" event={"ID":"936bcf63-8ee2-461b-8e1e-783a3c0ac208","Type":"ContainerStarted","Data":"c383f311c6233272671d09b3bad4ed7a33809ac0b3ae94dfb186348c2db00134"} Jul 29 20:26:42.509100 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:42.508932 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7" event={"ID":"936bcf63-8ee2-461b-8e1e-783a3c0ac208","Type":"ContainerStarted","Data":"20b4f21e71fd2f016b9da58d36b78e45651e0c130fae311d4118a5d249001eb6"} Jul 29 20:26:42.509100 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:42.508967 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7" event={"ID":"936bcf63-8ee2-461b-8e1e-783a3c0ac208","Type":"ContainerStarted","Data":"9c39eae7859b05f0735f57fb272b541931a2d01a1878e7507e6077bab7633ad9"} Jul 29 20:26:43.423832 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:43.423803 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-647c86dff8-ls4m7" Jul 29 20:26:43.516870 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:43.516834 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7" event={"ID":"936bcf63-8ee2-461b-8e1e-783a3c0ac208","Type":"ContainerStarted","Data":"9d9366f83a0f8c9d1da254e2d6e61c14a77101038e57329dda5664c82f31f89a"} Jul 29 20:26:43.518694 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:43.518599 2559 generic.go:356] "Generic (PLEG): container finished" podID="5797ad79-9ff2-4480-b284-ad4ce5000915" containerID="8ecf6822121cdeddba6e6b89312e4b1517e524eb58d26b3be6857b340408fea4" exitCode=0 Jul 29 20:26:43.518694 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:43.518668 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f5blf" event={"ID":"5797ad79-9ff2-4480-b284-ad4ce5000915","Type":"ContainerDied","Data":"8ecf6822121cdeddba6e6b89312e4b1517e524eb58d26b3be6857b340408fea4"} Jul 29 20:26:43.538507 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:43.538422 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-5bcs7" podStartSLOduration=2.295847857 podStartE2EDuration="3.538403607s" totalImagesPullingTime="1.24255575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:26:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:26:43.537394782 +0000 UTC m=+99.052701034" watchObservedRunningTime="2026-07-29 20:26:43.538403607 +0000 UTC m=+99.053709832" Jul 29 20:26:45.208399 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.208362 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-b46b855dd-4wd56"] Jul 29 20:26:45.211536 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.211517 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.215187 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.215161 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-nxrmq\"" Jul 29 20:26:45.215313 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.215195 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 29 20:26:45.215313 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.215212 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3r8nma5pl8fi4\"" Jul 29 20:26:45.215313 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.215198 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 29 20:26:45.215465 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.215314 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 29 20:26:45.215583 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.215161 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 29 20:26:45.225163 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.225134 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-b46b855dd-4wd56"] Jul 29 20:26:45.322728 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.322698 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/35a9a79c-5aeb-48f9-86b9-410263ba6c28-secret-metrics-server-client-certs\") pod \"metrics-server-b46b855dd-4wd56\" (UID: \"35a9a79c-5aeb-48f9-86b9-410263ba6c28\") " pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.322728 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.322730 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwbcl\" (UniqueName: \"kubernetes.io/projected/35a9a79c-5aeb-48f9-86b9-410263ba6c28-kube-api-access-dwbcl\") pod \"metrics-server-b46b855dd-4wd56\" (UID: \"35a9a79c-5aeb-48f9-86b9-410263ba6c28\") " pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.322934 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.322749 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/35a9a79c-5aeb-48f9-86b9-410263ba6c28-metrics-server-audit-profiles\") pod \"metrics-server-b46b855dd-4wd56\" (UID: \"35a9a79c-5aeb-48f9-86b9-410263ba6c28\") " pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.322934 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.322773 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35a9a79c-5aeb-48f9-86b9-410263ba6c28-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b46b855dd-4wd56\" (UID: \"35a9a79c-5aeb-48f9-86b9-410263ba6c28\") " pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.322934 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.322879 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/35a9a79c-5aeb-48f9-86b9-410263ba6c28-client-ca-bundle\") pod \"metrics-server-b46b855dd-4wd56\" (UID: \"35a9a79c-5aeb-48f9-86b9-410263ba6c28\") " pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.322934 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.322919 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/35a9a79c-5aeb-48f9-86b9-410263ba6c28-secret-metrics-server-tls\") pod \"metrics-server-b46b855dd-4wd56\" (UID: \"35a9a79c-5aeb-48f9-86b9-410263ba6c28\") " pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.323105 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.322947 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/35a9a79c-5aeb-48f9-86b9-410263ba6c28-audit-log\") pod \"metrics-server-b46b855dd-4wd56\" (UID: \"35a9a79c-5aeb-48f9-86b9-410263ba6c28\") " pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.423989 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.423889 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/35a9a79c-5aeb-48f9-86b9-410263ba6c28-secret-metrics-server-client-certs\") pod \"metrics-server-b46b855dd-4wd56\" (UID: \"35a9a79c-5aeb-48f9-86b9-410263ba6c28\") " pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.423989 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.423932 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwbcl\" (UniqueName: \"kubernetes.io/projected/35a9a79c-5aeb-48f9-86b9-410263ba6c28-kube-api-access-dwbcl\") pod \"metrics-server-b46b855dd-4wd56\" (UID: \"35a9a79c-5aeb-48f9-86b9-410263ba6c28\") " pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.423989 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.423957 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/35a9a79c-5aeb-48f9-86b9-410263ba6c28-metrics-server-audit-profiles\") pod \"metrics-server-b46b855dd-4wd56\" (UID: \"35a9a79c-5aeb-48f9-86b9-410263ba6c28\") " pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.423989 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.423988 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35a9a79c-5aeb-48f9-86b9-410263ba6c28-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b46b855dd-4wd56\" (UID: \"35a9a79c-5aeb-48f9-86b9-410263ba6c28\") " pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.424354 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.424086 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/35a9a79c-5aeb-48f9-86b9-410263ba6c28-client-ca-bundle\") pod \"metrics-server-b46b855dd-4wd56\" (UID: \"35a9a79c-5aeb-48f9-86b9-410263ba6c28\") " pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.424354 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.424125 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/35a9a79c-5aeb-48f9-86b9-410263ba6c28-secret-metrics-server-tls\") pod \"metrics-server-b46b855dd-4wd56\" (UID: \"35a9a79c-5aeb-48f9-86b9-410263ba6c28\") " pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.424354 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.424192 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/35a9a79c-5aeb-48f9-86b9-410263ba6c28-audit-log\") pod \"metrics-server-b46b855dd-4wd56\" (UID: \"35a9a79c-5aeb-48f9-86b9-410263ba6c28\") " pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.424568 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.424539 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/35a9a79c-5aeb-48f9-86b9-410263ba6c28-audit-log\") pod \"metrics-server-b46b855dd-4wd56\" (UID: \"35a9a79c-5aeb-48f9-86b9-410263ba6c28\") " pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.424848 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.424824 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35a9a79c-5aeb-48f9-86b9-410263ba6c28-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b46b855dd-4wd56\" (UID: \"35a9a79c-5aeb-48f9-86b9-410263ba6c28\") " pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.424958 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.424941 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/35a9a79c-5aeb-48f9-86b9-410263ba6c28-metrics-server-audit-profiles\") pod \"metrics-server-b46b855dd-4wd56\" (UID: \"35a9a79c-5aeb-48f9-86b9-410263ba6c28\") " pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.426524 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.426500 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/35a9a79c-5aeb-48f9-86b9-410263ba6c28-secret-metrics-server-client-certs\") pod \"metrics-server-b46b855dd-4wd56\" (UID: \"35a9a79c-5aeb-48f9-86b9-410263ba6c28\") " pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.426700 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.426678 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/35a9a79c-5aeb-48f9-86b9-410263ba6c28-secret-metrics-server-tls\") pod \"metrics-server-b46b855dd-4wd56\" (UID: \"35a9a79c-5aeb-48f9-86b9-410263ba6c28\") " pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.426761 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.426686 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/35a9a79c-5aeb-48f9-86b9-410263ba6c28-client-ca-bundle\") pod \"metrics-server-b46b855dd-4wd56\" (UID: \"35a9a79c-5aeb-48f9-86b9-410263ba6c28\") " pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.439748 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.439718 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwbcl\" (UniqueName: \"kubernetes.io/projected/35a9a79c-5aeb-48f9-86b9-410263ba6c28-kube-api-access-dwbcl\") pod \"metrics-server-b46b855dd-4wd56\" (UID: \"35a9a79c-5aeb-48f9-86b9-410263ba6c28\") " pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.529285 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.529243 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f5blf" event={"ID":"5797ad79-9ff2-4480-b284-ad4ce5000915","Type":"ContainerStarted","Data":"ea52ac93699e7879a1a03ca26d97341cda4b4cd43baccde139dfa80273170194"} Jul 29 20:26:45.529463 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.529300 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f5blf" event={"ID":"5797ad79-9ff2-4480-b284-ad4ce5000915","Type":"ContainerStarted","Data":"bebd945d8fefcf2347f1ba679037e4cd80ff3c33733f498b298b1875383c785a"} Jul 29 20:26:45.531285 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.531263 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dw8kz" event={"ID":"3e7cc3ba-05fb-4d6a-9211-21196d16d214","Type":"ContainerStarted","Data":"2589c8406d7b8cb01f75eef3fd4ceb28012e89a8d0783a5178a6227af86da718"} Jul 29 20:26:45.550003 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.549975 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-zrmw4"] Jul 29 20:26:45.554472 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.554437 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-zrmw4" Jul 29 20:26:45.557176 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.557153 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 29 20:26:45.557293 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.557201 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-vwzt8\"" Jul 29 20:26:45.557777 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.557760 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:26:45.562489 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.562440 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-f5blf" podStartSLOduration=4.266587208 podStartE2EDuration="5.562419893s" totalImagesPullingTime="1.213613905s" totalInitContainerRuntime="82.21878ms" isStatefulPod=true podCreationTimestamp="2026-07-29 20:26:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:26:45.561279776 +0000 UTC m=+101.076586002" watchObservedRunningTime="2026-07-29 20:26:45.562419893 +0000 UTC m=+101.077726118" Jul 29 20:26:45.565531 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.565509 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-zrmw4"] Jul 29 20:26:45.698072 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.697944 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dw8kz" podStartSLOduration=33.36993033 podStartE2EDuration="45.697924758s" totalImagesPullingTime="12.327994428s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:26:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:26:45.604555698 +0000 UTC m=+101.119861948" watchObservedRunningTime="2026-07-29 20:26:45.697924758 +0000 UTC m=+101.213230982" Jul 29 20:26:45.698378 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.698358 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-b46b855dd-4wd56"] Jul 29 20:26:45.701608 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:26:45.701579 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35a9a79c_5aeb_48f9_86b9_410263ba6c28.slice/crio-40430d652586a54d4248357723258478695ea335267a0843f2fb9863ca96c49f WatchSource:0}: Error finding container 40430d652586a54d4248357723258478695ea335267a0843f2fb9863ca96c49f: Status 404 returned error can't find the container with id 40430d652586a54d4248357723258478695ea335267a0843f2fb9863ca96c49f Jul 29 20:26:45.727196 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.727167 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rzgd\" (UniqueName: \"kubernetes.io/projected/74f7a0e9-1364-4196-846e-d936c3fa4c7b-kube-api-access-5rzgd\") pod \"monitoring-plugin-68b7fc9574-zrmw4\" (UID: \"74f7a0e9-1364-4196-846e-d936c3fa4c7b\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-zrmw4" Jul 29 20:26:45.727196 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.727204 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/74f7a0e9-1364-4196-846e-d936c3fa4c7b-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-zrmw4\" (UID: \"74f7a0e9-1364-4196-846e-d936c3fa4c7b\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-zrmw4" Jul 29 20:26:45.828291 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.828253 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rzgd\" (UniqueName: \"kubernetes.io/projected/74f7a0e9-1364-4196-846e-d936c3fa4c7b-kube-api-access-5rzgd\") pod \"monitoring-plugin-68b7fc9574-zrmw4\" (UID: \"74f7a0e9-1364-4196-846e-d936c3fa4c7b\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-zrmw4" Jul 29 20:26:45.828291 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.828289 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/74f7a0e9-1364-4196-846e-d936c3fa4c7b-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-zrmw4\" (UID: \"74f7a0e9-1364-4196-846e-d936c3fa4c7b\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-zrmw4" Jul 29 20:26:45.830778 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.830746 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/74f7a0e9-1364-4196-846e-d936c3fa4c7b-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-zrmw4\" (UID: \"74f7a0e9-1364-4196-846e-d936c3fa4c7b\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-zrmw4" Jul 29 20:26:45.838905 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.838879 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rzgd\" (UniqueName: \"kubernetes.io/projected/74f7a0e9-1364-4196-846e-d936c3fa4c7b-kube-api-access-5rzgd\") pod \"monitoring-plugin-68b7fc9574-zrmw4\" (UID: \"74f7a0e9-1364-4196-846e-d936c3fa4c7b\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-zrmw4" Jul 29 20:26:45.868767 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.868738 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-zrmw4" Jul 29 20:26:45.998024 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:45.997995 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-zrmw4"] Jul 29 20:26:46.001284 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:26:46.001256 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74f7a0e9_1364_4196_846e_d936c3fa4c7b.slice/crio-7582a225a648c6a3d005da208deb4d02ad6b012bcc731741cb4ff277074384cc WatchSource:0}: Error finding container 7582a225a648c6a3d005da208deb4d02ad6b012bcc731741cb4ff277074384cc: Status 404 returned error can't find the container with id 7582a225a648c6a3d005da208deb4d02ad6b012bcc731741cb4ff277074384cc Jul 29 20:26:46.026321 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.026292 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc"] Jul 29 20:26:46.030863 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.030842 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.033340 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.033315 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 29 20:26:46.033459 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.033381 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 29 20:26:46.033523 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.033465 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-q8ztf\"" Jul 29 20:26:46.033573 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.033532 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 29 20:26:46.033662 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.033645 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 29 20:26:46.033708 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.033658 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 29 20:26:46.038500 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.038477 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 29 20:26:46.043077 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.043033 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc"] Jul 29 20:26:46.131943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.131910 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/85617a10-471f-47a7-8fa8-31f7ced84817-secret-telemeter-client\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.131943 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.131948 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4xs5\" (UniqueName: \"kubernetes.io/projected/85617a10-471f-47a7-8fa8-31f7ced84817-kube-api-access-b4xs5\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.132198 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.131973 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85617a10-471f-47a7-8fa8-31f7ced84817-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.132198 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.132076 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/85617a10-471f-47a7-8fa8-31f7ced84817-telemeter-client-tls\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.132198 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.132137 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/85617a10-471f-47a7-8fa8-31f7ced84817-federate-client-tls\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.132198 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.132189 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85617a10-471f-47a7-8fa8-31f7ced84817-serving-certs-ca-bundle\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.132375 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.132247 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85617a10-471f-47a7-8fa8-31f7ced84817-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.132375 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.132268 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85617a10-471f-47a7-8fa8-31f7ced84817-metrics-client-ca\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.232753 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.232663 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/85617a10-471f-47a7-8fa8-31f7ced84817-secret-telemeter-client\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.232753 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.232706 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4xs5\" (UniqueName: \"kubernetes.io/projected/85617a10-471f-47a7-8fa8-31f7ced84817-kube-api-access-b4xs5\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.232753 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.232729 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85617a10-471f-47a7-8fa8-31f7ced84817-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.233339 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.232785 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/85617a10-471f-47a7-8fa8-31f7ced84817-telemeter-client-tls\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.233339 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.232818 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/85617a10-471f-47a7-8fa8-31f7ced84817-federate-client-tls\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.233339 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.232858 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85617a10-471f-47a7-8fa8-31f7ced84817-serving-certs-ca-bundle\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.233339 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.232901 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85617a10-471f-47a7-8fa8-31f7ced84817-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.233339 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.232924 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85617a10-471f-47a7-8fa8-31f7ced84817-metrics-client-ca\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.233872 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.233846 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85617a10-471f-47a7-8fa8-31f7ced84817-metrics-client-ca\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.234957 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.234927 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85617a10-471f-47a7-8fa8-31f7ced84817-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.235646 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.235439 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85617a10-471f-47a7-8fa8-31f7ced84817-serving-certs-ca-bundle\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.235900 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.235880 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/85617a10-471f-47a7-8fa8-31f7ced84817-telemeter-client-tls\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.236027 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.236002 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85617a10-471f-47a7-8fa8-31f7ced84817-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.236176 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.236128 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/85617a10-471f-47a7-8fa8-31f7ced84817-secret-telemeter-client\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.236176 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.236154 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/85617a10-471f-47a7-8fa8-31f7ced84817-federate-client-tls\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.245754 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.245729 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4xs5\" (UniqueName: \"kubernetes.io/projected/85617a10-471f-47a7-8fa8-31f7ced84817-kube-api-access-b4xs5\") pod \"telemeter-client-85fdbbd49c-7zqpc\" (UID: \"85617a10-471f-47a7-8fa8-31f7ced84817\") " pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.340478 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.340433 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" Jul 29 20:26:46.494607 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.494562 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc"] Jul 29 20:26:46.499008 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:26:46.498980 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85617a10_471f_47a7_8fa8_31f7ced84817.slice/crio-64d191e29de129f8c813c9baad8e0f702fc2e7aa2a109d98daa4a3812193bb50 WatchSource:0}: Error finding container 64d191e29de129f8c813c9baad8e0f702fc2e7aa2a109d98daa4a3812193bb50: Status 404 returned error can't find the container with id 64d191e29de129f8c813c9baad8e0f702fc2e7aa2a109d98daa4a3812193bb50 Jul 29 20:26:46.537063 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.537009 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-zrmw4" event={"ID":"74f7a0e9-1364-4196-846e-d936c3fa4c7b","Type":"ContainerStarted","Data":"7582a225a648c6a3d005da208deb4d02ad6b012bcc731741cb4ff277074384cc"} Jul 29 20:26:46.538417 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.538385 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" event={"ID":"35a9a79c-5aeb-48f9-86b9-410263ba6c28","Type":"ContainerStarted","Data":"40430d652586a54d4248357723258478695ea335267a0843f2fb9863ca96c49f"} Jul 29 20:26:46.539645 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.539616 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" event={"ID":"85617a10-471f-47a7-8fa8-31f7ced84817","Type":"ContainerStarted","Data":"64d191e29de129f8c813c9baad8e0f702fc2e7aa2a109d98daa4a3812193bb50"} Jul 29 20:26:46.789005 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:46.788709 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-555697c5b6-ljftx" podUID="7fa264d5-fcc9-4972-b37e-18fc7697e367" containerName="registry" containerID="cri-o://feae1006f76e76822edd11d4510f00058a514ea4752a3c7a7ea65db6250fd4ce" gracePeriod=30 Jul 29 20:26:47.548666 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:47.548629 2559 generic.go:356] "Generic (PLEG): container finished" podID="7fa264d5-fcc9-4972-b37e-18fc7697e367" containerID="feae1006f76e76822edd11d4510f00058a514ea4752a3c7a7ea65db6250fd4ce" exitCode=0 Jul 29 20:26:47.549173 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:47.548700 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-555697c5b6-ljftx" event={"ID":"7fa264d5-fcc9-4972-b37e-18fc7697e367","Type":"ContainerDied","Data":"feae1006f76e76822edd11d4510f00058a514ea4752a3c7a7ea65db6250fd4ce"} Jul 29 20:26:47.941463 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:47.941436 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:26:47.950257 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:47.950228 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7fa264d5-fcc9-4972-b37e-18fc7697e367-image-registry-private-configuration\") pod \"7fa264d5-fcc9-4972-b37e-18fc7697e367\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " Jul 29 20:26:47.950398 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:47.950276 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7fa264d5-fcc9-4972-b37e-18fc7697e367-trusted-ca\") pod \"7fa264d5-fcc9-4972-b37e-18fc7697e367\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " Jul 29 20:26:47.950398 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:47.950307 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-certificates\") pod \"7fa264d5-fcc9-4972-b37e-18fc7697e367\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " Jul 29 20:26:47.950483 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:47.950407 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-tls\") pod \"7fa264d5-fcc9-4972-b37e-18fc7697e367\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " Jul 29 20:26:47.950483 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:47.950447 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7fa264d5-fcc9-4972-b37e-18fc7697e367-ca-trust-extracted\") pod \"7fa264d5-fcc9-4972-b37e-18fc7697e367\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " Jul 29 20:26:47.950483 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:47.950474 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-bound-sa-token\") pod \"7fa264d5-fcc9-4972-b37e-18fc7697e367\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " Jul 29 20:26:47.950602 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:47.950505 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fk4vw\" (UniqueName: \"kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-kube-api-access-fk4vw\") pod \"7fa264d5-fcc9-4972-b37e-18fc7697e367\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " Jul 29 20:26:47.950602 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:47.950579 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7fa264d5-fcc9-4972-b37e-18fc7697e367-installation-pull-secrets\") pod \"7fa264d5-fcc9-4972-b37e-18fc7697e367\" (UID: \"7fa264d5-fcc9-4972-b37e-18fc7697e367\") " Jul 29 20:26:47.950725 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:47.950693 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "7fa264d5-fcc9-4972-b37e-18fc7697e367" (UID: "7fa264d5-fcc9-4972-b37e-18fc7697e367"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 20:26:47.950825 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:47.950803 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fa264d5-fcc9-4972-b37e-18fc7697e367-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "7fa264d5-fcc9-4972-b37e-18fc7697e367" (UID: "7fa264d5-fcc9-4972-b37e-18fc7697e367"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 20:26:47.951022 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:47.951003 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7fa264d5-fcc9-4972-b37e-18fc7697e367-trusted-ca\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:26:47.951104 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:47.951030 2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-certificates\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:26:47.952935 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:47.952887 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-kube-api-access-fk4vw" (OuterVolumeSpecName: "kube-api-access-fk4vw") pod "7fa264d5-fcc9-4972-b37e-18fc7697e367" (UID: "7fa264d5-fcc9-4972-b37e-18fc7697e367"). InnerVolumeSpecName "kube-api-access-fk4vw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 20:26:47.952935 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:47.952928 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fa264d5-fcc9-4972-b37e-18fc7697e367-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "7fa264d5-fcc9-4972-b37e-18fc7697e367" (UID: "7fa264d5-fcc9-4972-b37e-18fc7697e367"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 20:26:47.953122 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:47.953009 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "7fa264d5-fcc9-4972-b37e-18fc7697e367" (UID: "7fa264d5-fcc9-4972-b37e-18fc7697e367"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 20:26:47.953122 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:47.953021 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fa264d5-fcc9-4972-b37e-18fc7697e367-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "7fa264d5-fcc9-4972-b37e-18fc7697e367" (UID: "7fa264d5-fcc9-4972-b37e-18fc7697e367"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 20:26:47.953202 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:47.953180 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7fa264d5-fcc9-4972-b37e-18fc7697e367" (UID: "7fa264d5-fcc9-4972-b37e-18fc7697e367"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 20:26:47.959672 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:47.959648 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fa264d5-fcc9-4972-b37e-18fc7697e367-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "7fa264d5-fcc9-4972-b37e-18fc7697e367" (UID: "7fa264d5-fcc9-4972-b37e-18fc7697e367"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:26:48.051827 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:48.051786 2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-bound-sa-token\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:26:48.051827 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:48.051827 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fk4vw\" (UniqueName: \"kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-kube-api-access-fk4vw\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:26:48.052081 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:48.051844 2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7fa264d5-fcc9-4972-b37e-18fc7697e367-installation-pull-secrets\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:26:48.052081 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:48.051860 2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7fa264d5-fcc9-4972-b37e-18fc7697e367-image-registry-private-configuration\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:26:48.052081 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:48.051876 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fa264d5-fcc9-4972-b37e-18fc7697e367-registry-tls\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:26:48.052081 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:48.051889 2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7fa264d5-fcc9-4972-b37e-18fc7697e367-ca-trust-extracted\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:26:48.401234 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:48.401192 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-n5546" Jul 29 20:26:48.555268 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:48.555234 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" event={"ID":"35a9a79c-5aeb-48f9-86b9-410263ba6c28","Type":"ContainerStarted","Data":"6b815b3030699bb7696d86ed33d57a3f7d2a77d870c6b4a66d43b76c311b3f61"} Jul 29 20:26:48.556448 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:48.556431 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-555697c5b6-ljftx" Jul 29 20:26:48.556518 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:48.556442 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-555697c5b6-ljftx" event={"ID":"7fa264d5-fcc9-4972-b37e-18fc7697e367","Type":"ContainerDied","Data":"5537b5a685842d2099a36c59e1fe1ca6fe16b9e9411c1efa47d7d38085b795b2"} Jul 29 20:26:48.556518 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:48.556496 2559 scope.go:117] "RemoveContainer" containerID="feae1006f76e76822edd11d4510f00058a514ea4752a3c7a7ea65db6250fd4ce" Jul 29 20:26:48.558102 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:48.558042 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-zrmw4" event={"ID":"74f7a0e9-1364-4196-846e-d936c3fa4c7b","Type":"ContainerStarted","Data":"611c071e120843cf18a85163a0c8c000c6ba87e0f122b89fb990d8d529337c36"} Jul 29 20:26:48.558332 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:48.558287 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-zrmw4" Jul 29 20:26:48.563758 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:48.563732 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-zrmw4" Jul 29 20:26:48.573476 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:48.573438 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" podStartSLOduration=1.175932971 podStartE2EDuration="3.573423614s" totalImagesPullingTime="2.397490643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:26:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:26:48.573156436 +0000 UTC m=+104.088462657" watchObservedRunningTime="2026-07-29 20:26:48.573423614 +0000 UTC m=+104.088729832" Jul 29 20:26:48.588441 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:48.588363 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-zrmw4" podStartSLOduration=1.486924556 podStartE2EDuration="3.588345755s" totalImagesPullingTime="2.101421199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:26:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:26:48.587916724 +0000 UTC m=+104.103222947" watchObservedRunningTime="2026-07-29 20:26:48.588345755 +0000 UTC m=+104.103651980" Jul 29 20:26:48.605346 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:48.605317 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-555697c5b6-ljftx"] Jul 29 20:26:48.609933 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:48.609908 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-555697c5b6-ljftx"] Jul 29 20:26:49.027186 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:49.027154 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fa264d5-fcc9-4972-b37e-18fc7697e367" path="/var/lib/kubelet/pods/7fa264d5-fcc9-4972-b37e-18fc7697e367/volumes" Jul 29 20:26:50.567858 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:50.567816 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" event={"ID":"85617a10-471f-47a7-8fa8-31f7ced84817","Type":"ContainerStarted","Data":"c4a8433ef63a121e3f5b1f6cd558e49b9d684865dcc56316d429cd8b8ca42044"} Jul 29 20:26:51.573304 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:51.573272 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" event={"ID":"85617a10-471f-47a7-8fa8-31f7ced84817","Type":"ContainerStarted","Data":"6b703f1c7ae0ff6cfdb29ca137ffd9683b0aaaa7d5bde6633cf3a0736fe4b56c"} Jul 29 20:26:51.573670 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:51.573328 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" event={"ID":"85617a10-471f-47a7-8fa8-31f7ced84817","Type":"ContainerStarted","Data":"24aad0b618ec75045935bbc71cbe242cd18d5775e917ef1e8796bf5e875550ad"} Jul 29 20:26:51.599683 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:51.599631 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-85fdbbd49c-7zqpc" podStartSLOduration=1.305659811 podStartE2EDuration="5.599612835s" totalImagesPullingTime="4.293953024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:26:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:26:51.598763552 +0000 UTC m=+107.114069799" watchObservedRunningTime="2026-07-29 20:26:51.599612835 +0000 UTC m=+107.114919060" Jul 29 20:26:55.231274 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:55.231241 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-lmtmp"] Jul 29 20:26:55.231652 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:55.231517 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fa264d5-fcc9-4972-b37e-18fc7697e367" containerName="registry" Jul 29 20:26:55.231652 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:55.231527 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fa264d5-fcc9-4972-b37e-18fc7697e367" containerName="registry" Jul 29 20:26:55.231652 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:55.231580 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fa264d5-fcc9-4972-b37e-18fc7697e367" containerName="registry" Jul 29 20:26:55.233437 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:55.233421 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-lmtmp" Jul 29 20:26:55.235831 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:55.235806 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-9mf44\"" Jul 29 20:26:55.236386 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:55.236368 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 29 20:26:55.236479 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:55.236458 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 29 20:26:55.243424 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:55.243404 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-lmtmp"] Jul 29 20:26:55.313173 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:55.313142 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqvx7\" (UniqueName: \"kubernetes.io/projected/00b36d47-033f-4ba9-a2a0-8c9e56ba4393-kube-api-access-rqvx7\") pod \"downloads-7c97dc4555-lmtmp\" (UID: \"00b36d47-033f-4ba9-a2a0-8c9e56ba4393\") " pod="openshift-console/downloads-7c97dc4555-lmtmp" Jul 29 20:26:55.414386 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:55.414343 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqvx7\" (UniqueName: \"kubernetes.io/projected/00b36d47-033f-4ba9-a2a0-8c9e56ba4393-kube-api-access-rqvx7\") pod \"downloads-7c97dc4555-lmtmp\" (UID: \"00b36d47-033f-4ba9-a2a0-8c9e56ba4393\") " pod="openshift-console/downloads-7c97dc4555-lmtmp" Jul 29 20:26:55.423327 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:55.423295 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqvx7\" (UniqueName: \"kubernetes.io/projected/00b36d47-033f-4ba9-a2a0-8c9e56ba4393-kube-api-access-rqvx7\") pod \"downloads-7c97dc4555-lmtmp\" (UID: \"00b36d47-033f-4ba9-a2a0-8c9e56ba4393\") " pod="openshift-console/downloads-7c97dc4555-lmtmp" Jul 29 20:26:55.542566 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:55.542539 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-lmtmp" Jul 29 20:26:55.669924 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:55.669891 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-lmtmp"] Jul 29 20:26:55.673118 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:26:55.673088 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00b36d47_033f_4ba9_a2a0_8c9e56ba4393.slice/crio-c550b713c880f4a1053f4f1aa22e8080f85f7b144b7a84611904de0869373dad WatchSource:0}: Error finding container c550b713c880f4a1053f4f1aa22e8080f85f7b144b7a84611904de0869373dad: Status 404 returned error can't find the container with id c550b713c880f4a1053f4f1aa22e8080f85f7b144b7a84611904de0869373dad Jul 29 20:26:56.595713 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:26:56.595662 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-lmtmp" event={"ID":"00b36d47-033f-4ba9-a2a0-8c9e56ba4393","Type":"ContainerStarted","Data":"c550b713c880f4a1053f4f1aa22e8080f85f7b144b7a84611904de0869373dad"} Jul 29 20:27:05.558813 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:05.558775 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:27:05.559236 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:05.558842 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:27:05.812535 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:05.812451 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cc56cf788-nbvlw"] Jul 29 20:27:05.815361 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:05.815339 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:05.818341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:05.818320 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 29 20:27:05.819033 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:05.819007 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 29 20:27:05.819175 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:05.819158 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 29 20:27:05.819241 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:05.819185 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-g2zcr\"" Jul 29 20:27:05.819296 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:05.819277 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 29 20:27:05.826124 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:05.826087 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 29 20:27:05.833480 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:05.833459 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cc56cf788-nbvlw"] Jul 29 20:27:05.911936 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:05.911888 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed9952cb-b462-4bea-bfda-bd8f12a48f26-oauth-serving-cert\") pod \"console-cc56cf788-nbvlw\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:05.911936 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:05.911939 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed9952cb-b462-4bea-bfda-bd8f12a48f26-console-serving-cert\") pod \"console-cc56cf788-nbvlw\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:05.912186 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:05.912079 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed9952cb-b462-4bea-bfda-bd8f12a48f26-console-config\") pod \"console-cc56cf788-nbvlw\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:05.912186 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:05.912134 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed9952cb-b462-4bea-bfda-bd8f12a48f26-service-ca\") pod \"console-cc56cf788-nbvlw\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:05.912271 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:05.912192 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed9952cb-b462-4bea-bfda-bd8f12a48f26-console-oauth-config\") pod \"console-cc56cf788-nbvlw\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:05.912271 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:05.912245 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzx52\" (UniqueName: \"kubernetes.io/projected/ed9952cb-b462-4bea-bfda-bd8f12a48f26-kube-api-access-lzx52\") pod \"console-cc56cf788-nbvlw\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:06.013370 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:06.013339 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzx52\" (UniqueName: \"kubernetes.io/projected/ed9952cb-b462-4bea-bfda-bd8f12a48f26-kube-api-access-lzx52\") pod \"console-cc56cf788-nbvlw\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:06.013370 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:06.013406 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed9952cb-b462-4bea-bfda-bd8f12a48f26-oauth-serving-cert\") pod \"console-cc56cf788-nbvlw\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:06.013370 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:06.013437 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed9952cb-b462-4bea-bfda-bd8f12a48f26-console-serving-cert\") pod \"console-cc56cf788-nbvlw\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:06.013370 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:06.013592 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed9952cb-b462-4bea-bfda-bd8f12a48f26-console-config\") pod \"console-cc56cf788-nbvlw\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:06.013370 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:06.013642 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed9952cb-b462-4bea-bfda-bd8f12a48f26-service-ca\") pod \"console-cc56cf788-nbvlw\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:06.013370 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:06.013675 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed9952cb-b462-4bea-bfda-bd8f12a48f26-console-oauth-config\") pod \"console-cc56cf788-nbvlw\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:06.014378 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:06.014339 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed9952cb-b462-4bea-bfda-bd8f12a48f26-oauth-serving-cert\") pod \"console-cc56cf788-nbvlw\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:06.014946 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:06.014920 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed9952cb-b462-4bea-bfda-bd8f12a48f26-service-ca\") pod \"console-cc56cf788-nbvlw\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:06.015076 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:06.014957 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed9952cb-b462-4bea-bfda-bd8f12a48f26-console-config\") pod \"console-cc56cf788-nbvlw\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:06.016343 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:06.016319 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed9952cb-b462-4bea-bfda-bd8f12a48f26-console-serving-cert\") pod \"console-cc56cf788-nbvlw\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:06.016433 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:06.016325 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed9952cb-b462-4bea-bfda-bd8f12a48f26-console-oauth-config\") pod \"console-cc56cf788-nbvlw\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:06.022535 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:06.022512 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzx52\" (UniqueName: \"kubernetes.io/projected/ed9952cb-b462-4bea-bfda-bd8f12a48f26-kube-api-access-lzx52\") pod \"console-cc56cf788-nbvlw\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:06.126028 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:06.125917 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:12.190811 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:12.190777 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cc56cf788-nbvlw"] Jul 29 20:27:12.277387 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:27:12.277355 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded9952cb_b462_4bea_bfda_bd8f12a48f26.slice/crio-953307def47fd3a825b9a8d88354f32032c1ea147afbecf5a9b8c4987e93e7ea WatchSource:0}: Error finding container 953307def47fd3a825b9a8d88354f32032c1ea147afbecf5a9b8c4987e93e7ea: Status 404 returned error can't find the container with id 953307def47fd3a825b9a8d88354f32032c1ea147afbecf5a9b8c4987e93e7ea Jul 29 20:27:12.663514 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:12.663427 2559 generic.go:356] "Generic (PLEG): container finished" podID="fc9b1faf-5638-4fd3-ae81-d579fbf2f56f" containerID="499fdd2ade8fd2219e19117ed3976308acf6c13abc0ea81f62493b639fa46d52" exitCode=0 Jul 29 20:27:12.663749 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:12.663511 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-s7t8m" event={"ID":"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f","Type":"ContainerDied","Data":"499fdd2ade8fd2219e19117ed3976308acf6c13abc0ea81f62493b639fa46d52"} Jul 29 20:27:12.663949 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:12.663922 2559 scope.go:117] "RemoveContainer" containerID="499fdd2ade8fd2219e19117ed3976308acf6c13abc0ea81f62493b639fa46d52" Jul 29 20:27:12.665372 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:12.665352 2559 generic.go:356] "Generic (PLEG): container finished" podID="2f8afdfb-3427-4d4d-b0a6-1061e4571298" containerID="900410dcc0052ec5df236268363d0510fce3f9e18b2556f83426db6e4531209c" exitCode=0 Jul 29 20:27:12.665471 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:12.665434 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-42vmx" event={"ID":"2f8afdfb-3427-4d4d-b0a6-1061e4571298","Type":"ContainerDied","Data":"900410dcc0052ec5df236268363d0510fce3f9e18b2556f83426db6e4531209c"} Jul 29 20:27:12.665740 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:12.665724 2559 scope.go:117] "RemoveContainer" containerID="900410dcc0052ec5df236268363d0510fce3f9e18b2556f83426db6e4531209c" Jul 29 20:27:12.666790 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:12.666765 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cc56cf788-nbvlw" event={"ID":"ed9952cb-b462-4bea-bfda-bd8f12a48f26","Type":"ContainerStarted","Data":"953307def47fd3a825b9a8d88354f32032c1ea147afbecf5a9b8c4987e93e7ea"} Jul 29 20:27:12.668345 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:12.668317 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-lmtmp" event={"ID":"00b36d47-033f-4ba9-a2a0-8c9e56ba4393","Type":"ContainerStarted","Data":"4c77d3bfd4454d68fbc0d2633f4c317eccdbf22f30110b1636edfac0c5d703af"} Jul 29 20:27:12.668626 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:12.668563 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-lmtmp" Jul 29 20:27:12.670106 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:12.670083 2559 patch_prober.go:28] interesting pod/downloads-7c97dc4555-lmtmp container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.20:8080/\": dial tcp 10.134.0.20:8080: connect: connection refused" start-of-body= Jul 29 20:27:12.670219 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:12.670129 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7c97dc4555-lmtmp" podUID="00b36d47-033f-4ba9-a2a0-8c9e56ba4393" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.20:8080/\": dial tcp 10.134.0.20:8080: connect: connection refused" Jul 29 20:27:12.723811 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:12.723755 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-lmtmp" podStartSLOduration=1.053507463 podStartE2EDuration="17.723734698s" totalImagesPullingTime="16.670227235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:26:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:27:12.723516223 +0000 UTC m=+128.238822470" watchObservedRunningTime="2026-07-29 20:27:12.723734698 +0000 UTC m=+128.239040923" Jul 29 20:27:13.676976 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:13.676221 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-s7t8m" event={"ID":"fc9b1faf-5638-4fd3-ae81-d579fbf2f56f","Type":"ContainerStarted","Data":"f071a4ed9d4015efa660ee899c4bd06889a191e0de86061a2bfd7b70fdf01e5c"} Jul 29 20:27:13.681846 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:13.680909 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-42vmx" event={"ID":"2f8afdfb-3427-4d4d-b0a6-1061e4571298","Type":"ContainerStarted","Data":"6a81629edbdd322ab56cf6c2a92edd9503ca1ad75fb4435ce37eafcce4ae9324"} Jul 29 20:27:13.697342 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:13.697287 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-lmtmp" Jul 29 20:27:14.802478 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:14.802399 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs\") pod \"network-metrics-daemon-vsmj9\" (UID: \"e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1\") " pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:27:14.805143 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:14.805119 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1-metrics-certs\") pod \"network-metrics-daemon-vsmj9\" (UID: \"e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1\") " pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:27:15.047936 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:15.047900 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f7tqs\"" Jul 29 20:27:15.055903 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:15.055835 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vsmj9" Jul 29 20:27:16.241932 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:16.241909 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vsmj9"] Jul 29 20:27:16.244172 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:27:16.244142 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8f8cbb6_aaf2_4e92_a7fe_351fadad61a1.slice/crio-42496190fefc4e1785843c00dd37b5b73156c1db43d44255ade2eea626cf70b4 WatchSource:0}: Error finding container 42496190fefc4e1785843c00dd37b5b73156c1db43d44255ade2eea626cf70b4: Status 404 returned error can't find the container with id 42496190fefc4e1785843c00dd37b5b73156c1db43d44255ade2eea626cf70b4 Jul 29 20:27:16.701968 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:16.701454 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cc56cf788-nbvlw" event={"ID":"ed9952cb-b462-4bea-bfda-bd8f12a48f26","Type":"ContainerStarted","Data":"f6d4242ffdf5f5c7bd72a70888b2ffbbd9ae94c8daf4e01e3d025224f6c28665"} Jul 29 20:27:16.702818 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:16.702781 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vsmj9" event={"ID":"e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1","Type":"ContainerStarted","Data":"42496190fefc4e1785843c00dd37b5b73156c1db43d44255ade2eea626cf70b4"} Jul 29 20:27:16.721654 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:16.721589 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cc56cf788-nbvlw" podStartSLOduration=7.486564667 podStartE2EDuration="11.72157054s" totalImagesPullingTime="4.235005873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:27:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:27:16.720663054 +0000 UTC m=+132.235969280" watchObservedRunningTime="2026-07-29 20:27:16.72157054 +0000 UTC m=+132.236876768" Jul 29 20:27:18.715178 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:18.715138 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vsmj9" event={"ID":"e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1","Type":"ContainerStarted","Data":"4d63e2f69ddc70a80e3782ceef37f6755b25f6443117c8faf4f95b75fe3447c5"} Jul 29 20:27:18.715693 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:18.715202 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vsmj9" event={"ID":"e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1","Type":"ContainerStarted","Data":"790959f32c05938cc5f7c28fa792b640010e4c5ffe306c8514b8665363f47f27"} Jul 29 20:27:18.732872 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:18.732816 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vsmj9" podStartSLOduration=132.318767796 podStartE2EDuration="2m13.732797157s" totalImagesPullingTime="1.414029361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:25:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:27:18.731479565 +0000 UTC m=+134.246785790" watchObservedRunningTime="2026-07-29 20:27:18.732797157 +0000 UTC m=+134.248103384" Jul 29 20:27:25.333672 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:25.333631 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cc56cf788-nbvlw"] Jul 29 20:27:25.563980 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:25.563949 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:27:25.567816 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:25.567790 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-b46b855dd-4wd56" Jul 29 20:27:26.126459 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:26.126430 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:50.353302 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.353243 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-cc56cf788-nbvlw" podUID="ed9952cb-b462-4bea-bfda-bd8f12a48f26" containerName="console" containerID="cri-o://f6d4242ffdf5f5c7bd72a70888b2ffbbd9ae94c8daf4e01e3d025224f6c28665" gracePeriod=15 Jul 29 20:27:50.616555 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.616499 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:50.688493 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.688466 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed9952cb-b462-4bea-bfda-bd8f12a48f26-service-ca\") pod \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " Jul 29 20:27:50.688685 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.688502 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzx52\" (UniqueName: \"kubernetes.io/projected/ed9952cb-b462-4bea-bfda-bd8f12a48f26-kube-api-access-lzx52\") pod \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " Jul 29 20:27:50.688685 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.688531 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed9952cb-b462-4bea-bfda-bd8f12a48f26-oauth-serving-cert\") pod \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " Jul 29 20:27:50.688685 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.688564 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed9952cb-b462-4bea-bfda-bd8f12a48f26-console-oauth-config\") pod \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " Jul 29 20:27:50.688685 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.688641 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed9952cb-b462-4bea-bfda-bd8f12a48f26-console-config\") pod \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " Jul 29 20:27:50.688685 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.688676 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed9952cb-b462-4bea-bfda-bd8f12a48f26-console-serving-cert\") pod \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\" (UID: \"ed9952cb-b462-4bea-bfda-bd8f12a48f26\") " Jul 29 20:27:50.689018 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.688978 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed9952cb-b462-4bea-bfda-bd8f12a48f26-service-ca" (OuterVolumeSpecName: "service-ca") pod "ed9952cb-b462-4bea-bfda-bd8f12a48f26" (UID: "ed9952cb-b462-4bea-bfda-bd8f12a48f26"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 20:27:50.689149 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.688987 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed9952cb-b462-4bea-bfda-bd8f12a48f26-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ed9952cb-b462-4bea-bfda-bd8f12a48f26" (UID: "ed9952cb-b462-4bea-bfda-bd8f12a48f26"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 20:27:50.689149 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.689016 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed9952cb-b462-4bea-bfda-bd8f12a48f26-console-config" (OuterVolumeSpecName: "console-config") pod "ed9952cb-b462-4bea-bfda-bd8f12a48f26" (UID: "ed9952cb-b462-4bea-bfda-bd8f12a48f26"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 20:27:50.690810 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.690781 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed9952cb-b462-4bea-bfda-bd8f12a48f26-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ed9952cb-b462-4bea-bfda-bd8f12a48f26" (UID: "ed9952cb-b462-4bea-bfda-bd8f12a48f26"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 20:27:50.690898 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.690804 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed9952cb-b462-4bea-bfda-bd8f12a48f26-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ed9952cb-b462-4bea-bfda-bd8f12a48f26" (UID: "ed9952cb-b462-4bea-bfda-bd8f12a48f26"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 20:27:50.690898 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.690884 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed9952cb-b462-4bea-bfda-bd8f12a48f26-kube-api-access-lzx52" (OuterVolumeSpecName: "kube-api-access-lzx52") pod "ed9952cb-b462-4bea-bfda-bd8f12a48f26" (UID: "ed9952cb-b462-4bea-bfda-bd8f12a48f26"). InnerVolumeSpecName "kube-api-access-lzx52". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 20:27:50.789732 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.789695 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed9952cb-b462-4bea-bfda-bd8f12a48f26-console-serving-cert\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:27:50.789732 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.789726 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed9952cb-b462-4bea-bfda-bd8f12a48f26-service-ca\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:27:50.789732 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.789736 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzx52\" (UniqueName: \"kubernetes.io/projected/ed9952cb-b462-4bea-bfda-bd8f12a48f26-kube-api-access-lzx52\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:27:50.789952 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.789745 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed9952cb-b462-4bea-bfda-bd8f12a48f26-oauth-serving-cert\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:27:50.789952 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.789754 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed9952cb-b462-4bea-bfda-bd8f12a48f26-console-oauth-config\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:27:50.789952 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.789762 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed9952cb-b462-4bea-bfda-bd8f12a48f26-console-config\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:27:50.829881 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.829845 2559 generic.go:356] "Generic (PLEG): container finished" podID="ed9952cb-b462-4bea-bfda-bd8f12a48f26" containerID="f6d4242ffdf5f5c7bd72a70888b2ffbbd9ae94c8daf4e01e3d025224f6c28665" exitCode=0 Jul 29 20:27:50.830005 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.829895 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cc56cf788-nbvlw" Jul 29 20:27:50.830005 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.829906 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cc56cf788-nbvlw" event={"ID":"ed9952cb-b462-4bea-bfda-bd8f12a48f26","Type":"ContainerDied","Data":"f6d4242ffdf5f5c7bd72a70888b2ffbbd9ae94c8daf4e01e3d025224f6c28665"} Jul 29 20:27:50.830005 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.829955 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cc56cf788-nbvlw" event={"ID":"ed9952cb-b462-4bea-bfda-bd8f12a48f26","Type":"ContainerDied","Data":"953307def47fd3a825b9a8d88354f32032c1ea147afbecf5a9b8c4987e93e7ea"} Jul 29 20:27:50.830005 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.829983 2559 scope.go:117] "RemoveContainer" containerID="f6d4242ffdf5f5c7bd72a70888b2ffbbd9ae94c8daf4e01e3d025224f6c28665" Jul 29 20:27:50.838379 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.838362 2559 scope.go:117] "RemoveContainer" containerID="f6d4242ffdf5f5c7bd72a70888b2ffbbd9ae94c8daf4e01e3d025224f6c28665" Jul 29 20:27:50.838635 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:27:50.838615 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6d4242ffdf5f5c7bd72a70888b2ffbbd9ae94c8daf4e01e3d025224f6c28665\": container with ID starting with f6d4242ffdf5f5c7bd72a70888b2ffbbd9ae94c8daf4e01e3d025224f6c28665 not found: ID does not exist" containerID="f6d4242ffdf5f5c7bd72a70888b2ffbbd9ae94c8daf4e01e3d025224f6c28665" Jul 29 20:27:50.838698 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.838643 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6d4242ffdf5f5c7bd72a70888b2ffbbd9ae94c8daf4e01e3d025224f6c28665"} err="failed to get container status \"f6d4242ffdf5f5c7bd72a70888b2ffbbd9ae94c8daf4e01e3d025224f6c28665\": rpc error: code = NotFound desc = could not find container \"f6d4242ffdf5f5c7bd72a70888b2ffbbd9ae94c8daf4e01e3d025224f6c28665\": container with ID starting with f6d4242ffdf5f5c7bd72a70888b2ffbbd9ae94c8daf4e01e3d025224f6c28665 not found: ID does not exist" Jul 29 20:27:50.849853 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.849828 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cc56cf788-nbvlw"] Jul 29 20:27:50.853527 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:50.853508 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cc56cf788-nbvlw"] Jul 29 20:27:51.024482 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:51.024453 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed9952cb-b462-4bea-bfda-bd8f12a48f26" path="/var/lib/kubelet/pods/ed9952cb-b462-4bea-bfda-bd8f12a48f26/volumes" Jul 29 20:27:57.278860 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.278779 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9c6bfddc8-j88hw"] Jul 29 20:27:57.279479 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.279085 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed9952cb-b462-4bea-bfda-bd8f12a48f26" containerName="console" Jul 29 20:27:57.279479 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.279097 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed9952cb-b462-4bea-bfda-bd8f12a48f26" containerName="console" Jul 29 20:27:57.279479 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.279148 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed9952cb-b462-4bea-bfda-bd8f12a48f26" containerName="console" Jul 29 20:27:57.299154 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.299127 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9c6bfddc8-j88hw"] Jul 29 20:27:57.299297 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.299257 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.301607 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.301584 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 29 20:27:57.301825 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.301812 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 29 20:27:57.301880 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.301835 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 29 20:27:57.302661 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.302644 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 29 20:27:57.302661 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.302655 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-g2zcr\"" Jul 29 20:27:57.302844 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.302726 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 29 20:27:57.307376 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.307360 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 29 20:27:57.348170 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.348137 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-console-config\") pod \"console-9c6bfddc8-j88hw\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.348339 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.348186 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a75e126-5067-462e-9a90-7f912831af8d-console-serving-cert\") pod \"console-9c6bfddc8-j88hw\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.348339 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.348216 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cdjs\" (UniqueName: \"kubernetes.io/projected/7a75e126-5067-462e-9a90-7f912831af8d-kube-api-access-8cdjs\") pod \"console-9c6bfddc8-j88hw\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.348339 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.348240 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-trusted-ca-bundle\") pod \"console-9c6bfddc8-j88hw\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.348339 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.348288 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-service-ca\") pod \"console-9c6bfddc8-j88hw\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.348339 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.348316 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a75e126-5067-462e-9a90-7f912831af8d-console-oauth-config\") pod \"console-9c6bfddc8-j88hw\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.348339 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.348335 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-oauth-serving-cert\") pod \"console-9c6bfddc8-j88hw\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.449037 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.449004 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-console-config\") pod \"console-9c6bfddc8-j88hw\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.449254 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.449125 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a75e126-5067-462e-9a90-7f912831af8d-console-serving-cert\") pod \"console-9c6bfddc8-j88hw\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.449254 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.449152 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cdjs\" (UniqueName: \"kubernetes.io/projected/7a75e126-5067-462e-9a90-7f912831af8d-kube-api-access-8cdjs\") pod \"console-9c6bfddc8-j88hw\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.449254 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.449182 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-trusted-ca-bundle\") pod \"console-9c6bfddc8-j88hw\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.449254 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.449237 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-service-ca\") pod \"console-9c6bfddc8-j88hw\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.449474 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.449264 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a75e126-5067-462e-9a90-7f912831af8d-console-oauth-config\") pod \"console-9c6bfddc8-j88hw\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.449474 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.449285 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-oauth-serving-cert\") pod \"console-9c6bfddc8-j88hw\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.449847 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.449817 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-console-config\") pod \"console-9c6bfddc8-j88hw\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.449965 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.449944 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-service-ca\") pod \"console-9c6bfddc8-j88hw\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.450006 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.449961 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-oauth-serving-cert\") pod \"console-9c6bfddc8-j88hw\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.450212 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.450189 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-trusted-ca-bundle\") pod \"console-9c6bfddc8-j88hw\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.451674 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.451657 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a75e126-5067-462e-9a90-7f912831af8d-console-oauth-config\") pod \"console-9c6bfddc8-j88hw\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.451834 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.451816 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a75e126-5067-462e-9a90-7f912831af8d-console-serving-cert\") pod \"console-9c6bfddc8-j88hw\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.458138 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.458117 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cdjs\" (UniqueName: \"kubernetes.io/projected/7a75e126-5067-462e-9a90-7f912831af8d-kube-api-access-8cdjs\") pod \"console-9c6bfddc8-j88hw\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.609118 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.608993 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:27:57.767280 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.767178 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9c6bfddc8-j88hw"] Jul 29 20:27:57.769907 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:27:57.769875 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a75e126_5067_462e_9a90_7f912831af8d.slice/crio-cfc4c73a9cc56f97a94cd0a63c53963dbb725f0145e7000e786f9eda20dcc8b0 WatchSource:0}: Error finding container cfc4c73a9cc56f97a94cd0a63c53963dbb725f0145e7000e786f9eda20dcc8b0: Status 404 returned error can't find the container with id cfc4c73a9cc56f97a94cd0a63c53963dbb725f0145e7000e786f9eda20dcc8b0 Jul 29 20:27:57.856194 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:57.856155 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9c6bfddc8-j88hw" event={"ID":"7a75e126-5067-462e-9a90-7f912831af8d","Type":"ContainerStarted","Data":"cfc4c73a9cc56f97a94cd0a63c53963dbb725f0145e7000e786f9eda20dcc8b0"} Jul 29 20:27:58.861705 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:58.861673 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9c6bfddc8-j88hw" event={"ID":"7a75e126-5067-462e-9a90-7f912831af8d","Type":"ContainerStarted","Data":"b748eadc7a848d49e73fb6487e6d302fde976c5fcf14ee413ba12af8f9ec5566"} Jul 29 20:27:58.881283 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:27:58.881232 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9c6bfddc8-j88hw" podStartSLOduration=1.881215753 podStartE2EDuration="1.881215753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:27:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:27:58.880123333 +0000 UTC m=+174.395429579" watchObservedRunningTime="2026-07-29 20:27:58.881215753 +0000 UTC m=+174.396521976" Jul 29 20:28:07.609434 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:28:07.609387 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:28:07.609869 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:28:07.609456 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:28:07.614275 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:28:07.614252 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:28:07.897755 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:28:07.897672 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:28:59.793934 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:28:59.793903 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk"] Jul 29 20:28:59.796036 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:28:59.796020 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk" Jul 29 20:28:59.798546 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:28:59.798527 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 29 20:28:59.799338 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:28:59.799317 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dhfxx\"" Jul 29 20:28:59.799338 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:28:59.799329 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 29 20:28:59.806405 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:28:59.806383 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk"] Jul 29 20:28:59.838599 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:28:59.838573 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fae2d861-652a-4488-aeb7-aa19defed2c1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk\" (UID: \"fae2d861-652a-4488-aeb7-aa19defed2c1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk" Jul 29 20:28:59.838705 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:28:59.838612 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fae2d861-652a-4488-aeb7-aa19defed2c1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk\" (UID: \"fae2d861-652a-4488-aeb7-aa19defed2c1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk" Jul 29 20:28:59.838705 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:28:59.838659 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bcrz\" (UniqueName: \"kubernetes.io/projected/fae2d861-652a-4488-aeb7-aa19defed2c1-kube-api-access-5bcrz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk\" (UID: \"fae2d861-652a-4488-aeb7-aa19defed2c1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk" Jul 29 20:28:59.940038 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:28:59.940009 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bcrz\" (UniqueName: \"kubernetes.io/projected/fae2d861-652a-4488-aeb7-aa19defed2c1-kube-api-access-5bcrz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk\" (UID: \"fae2d861-652a-4488-aeb7-aa19defed2c1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk" Jul 29 20:28:59.940184 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:28:59.940082 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fae2d861-652a-4488-aeb7-aa19defed2c1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk\" (UID: \"fae2d861-652a-4488-aeb7-aa19defed2c1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk" Jul 29 20:28:59.940184 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:28:59.940132 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fae2d861-652a-4488-aeb7-aa19defed2c1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk\" (UID: \"fae2d861-652a-4488-aeb7-aa19defed2c1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk" Jul 29 20:28:59.940446 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:28:59.940427 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fae2d861-652a-4488-aeb7-aa19defed2c1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk\" (UID: \"fae2d861-652a-4488-aeb7-aa19defed2c1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk" Jul 29 20:28:59.940493 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:28:59.940477 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fae2d861-652a-4488-aeb7-aa19defed2c1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk\" (UID: \"fae2d861-652a-4488-aeb7-aa19defed2c1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk" Jul 29 20:28:59.948210 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:28:59.948185 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bcrz\" (UniqueName: \"kubernetes.io/projected/fae2d861-652a-4488-aeb7-aa19defed2c1-kube-api-access-5bcrz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk\" (UID: \"fae2d861-652a-4488-aeb7-aa19defed2c1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk" Jul 29 20:29:00.104541 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:00.104463 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk" Jul 29 20:29:00.226399 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:00.226359 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk"] Jul 29 20:29:00.229257 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:29:00.229230 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfae2d861_652a_4488_aeb7_aa19defed2c1.slice/crio-701d341b9527b617a01b27dad2b892623687c7bf95a5d65486c6d6f7d6d71f7a WatchSource:0}: Error finding container 701d341b9527b617a01b27dad2b892623687c7bf95a5d65486c6d6f7d6d71f7a: Status 404 returned error can't find the container with id 701d341b9527b617a01b27dad2b892623687c7bf95a5d65486c6d6f7d6d71f7a Jul 29 20:29:01.095323 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:01.095290 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk" event={"ID":"fae2d861-652a-4488-aeb7-aa19defed2c1","Type":"ContainerStarted","Data":"701d341b9527b617a01b27dad2b892623687c7bf95a5d65486c6d6f7d6d71f7a"} Jul 29 20:29:09.129246 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:09.129210 2559 generic.go:356] "Generic (PLEG): container finished" podID="fae2d861-652a-4488-aeb7-aa19defed2c1" containerID="f452361572d1ec8066212960c740ea77053d113980c964c304dc51ad26657d4a" exitCode=0 Jul 29 20:29:09.129824 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:09.129290 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk" event={"ID":"fae2d861-652a-4488-aeb7-aa19defed2c1","Type":"ContainerDied","Data":"f452361572d1ec8066212960c740ea77053d113980c964c304dc51ad26657d4a"} Jul 29 20:29:12.143193 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:12.143157 2559 generic.go:356] "Generic (PLEG): container finished" podID="fae2d861-652a-4488-aeb7-aa19defed2c1" containerID="ce614318607bc8064d791acbbcc019d89fbf724a13a3c25be2a22c9cc493c2e4" exitCode=0 Jul 29 20:29:12.143564 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:12.143239 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk" event={"ID":"fae2d861-652a-4488-aeb7-aa19defed2c1","Type":"ContainerDied","Data":"ce614318607bc8064d791acbbcc019d89fbf724a13a3c25be2a22c9cc493c2e4"} Jul 29 20:29:19.172901 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:19.172867 2559 generic.go:356] "Generic (PLEG): container finished" podID="fae2d861-652a-4488-aeb7-aa19defed2c1" containerID="b1f0e60db274bafc5f203b5408de9f444818d0632e4fc7aa80d0901fab61986f" exitCode=0 Jul 29 20:29:19.173341 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:19.172938 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk" event={"ID":"fae2d861-652a-4488-aeb7-aa19defed2c1","Type":"ContainerDied","Data":"b1f0e60db274bafc5f203b5408de9f444818d0632e4fc7aa80d0901fab61986f"} Jul 29 20:29:20.300670 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:20.300651 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk" Jul 29 20:29:20.425637 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:20.425608 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fae2d861-652a-4488-aeb7-aa19defed2c1-bundle\") pod \"fae2d861-652a-4488-aeb7-aa19defed2c1\" (UID: \"fae2d861-652a-4488-aeb7-aa19defed2c1\") " Jul 29 20:29:20.425802 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:20.425651 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bcrz\" (UniqueName: \"kubernetes.io/projected/fae2d861-652a-4488-aeb7-aa19defed2c1-kube-api-access-5bcrz\") pod \"fae2d861-652a-4488-aeb7-aa19defed2c1\" (UID: \"fae2d861-652a-4488-aeb7-aa19defed2c1\") " Jul 29 20:29:20.425802 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:20.425699 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fae2d861-652a-4488-aeb7-aa19defed2c1-util\") pod \"fae2d861-652a-4488-aeb7-aa19defed2c1\" (UID: \"fae2d861-652a-4488-aeb7-aa19defed2c1\") " Jul 29 20:29:20.426282 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:20.426256 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fae2d861-652a-4488-aeb7-aa19defed2c1-bundle" (OuterVolumeSpecName: "bundle") pod "fae2d861-652a-4488-aeb7-aa19defed2c1" (UID: "fae2d861-652a-4488-aeb7-aa19defed2c1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:29:20.427905 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:20.427851 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fae2d861-652a-4488-aeb7-aa19defed2c1-kube-api-access-5bcrz" (OuterVolumeSpecName: "kube-api-access-5bcrz") pod "fae2d861-652a-4488-aeb7-aa19defed2c1" (UID: "fae2d861-652a-4488-aeb7-aa19defed2c1"). InnerVolumeSpecName "kube-api-access-5bcrz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 20:29:20.429793 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:20.429771 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fae2d861-652a-4488-aeb7-aa19defed2c1-util" (OuterVolumeSpecName: "util") pod "fae2d861-652a-4488-aeb7-aa19defed2c1" (UID: "fae2d861-652a-4488-aeb7-aa19defed2c1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:29:20.526833 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:20.526810 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fae2d861-652a-4488-aeb7-aa19defed2c1-bundle\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:29:20.526833 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:20.526831 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bcrz\" (UniqueName: \"kubernetes.io/projected/fae2d861-652a-4488-aeb7-aa19defed2c1-kube-api-access-5bcrz\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:29:20.526965 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:20.526840 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fae2d861-652a-4488-aeb7-aa19defed2c1-util\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:29:21.182212 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:21.182186 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk" Jul 29 20:29:21.182387 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:21.182178 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406p9qk" event={"ID":"fae2d861-652a-4488-aeb7-aa19defed2c1","Type":"ContainerDied","Data":"701d341b9527b617a01b27dad2b892623687c7bf95a5d65486c6d6f7d6d71f7a"} Jul 29 20:29:21.182387 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:21.182320 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="701d341b9527b617a01b27dad2b892623687c7bf95a5d65486c6d6f7d6d71f7a" Jul 29 20:29:27.836467 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:27.836388 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dtb6s"] Jul 29 20:29:27.836795 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:27.836651 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fae2d861-652a-4488-aeb7-aa19defed2c1" containerName="pull" Jul 29 20:29:27.836795 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:27.836662 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fae2d861-652a-4488-aeb7-aa19defed2c1" containerName="pull" Jul 29 20:29:27.836795 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:27.836673 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fae2d861-652a-4488-aeb7-aa19defed2c1" containerName="extract" Jul 29 20:29:27.836795 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:27.836679 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fae2d861-652a-4488-aeb7-aa19defed2c1" containerName="extract" Jul 29 20:29:27.836795 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:27.836688 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fae2d861-652a-4488-aeb7-aa19defed2c1" containerName="util" Jul 29 20:29:27.836795 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:27.836694 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fae2d861-652a-4488-aeb7-aa19defed2c1" containerName="util" Jul 29 20:29:27.836795 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:27.836748 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fae2d861-652a-4488-aeb7-aa19defed2c1" containerName="extract" Jul 29 20:29:27.838609 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:27.838593 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dtb6s" Jul 29 20:29:27.840981 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:27.840954 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 29 20:29:27.841164 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:27.841148 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-ggjdx\"" Jul 29 20:29:27.841241 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:27.841152 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 29 20:29:27.852168 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:27.852148 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dtb6s"] Jul 29 20:29:27.984286 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:27.984250 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n57b9\" (UniqueName: \"kubernetes.io/projected/4c203790-6f9a-4aab-ae7e-a39419fe343f-kube-api-access-n57b9\") pod \"cert-manager-operator-controller-manager-55c444bf86-dtb6s\" (UID: \"4c203790-6f9a-4aab-ae7e-a39419fe343f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dtb6s" Jul 29 20:29:27.984451 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:27.984316 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c203790-6f9a-4aab-ae7e-a39419fe343f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-dtb6s\" (UID: \"4c203790-6f9a-4aab-ae7e-a39419fe343f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dtb6s" Jul 29 20:29:28.085670 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:28.085622 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c203790-6f9a-4aab-ae7e-a39419fe343f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-dtb6s\" (UID: \"4c203790-6f9a-4aab-ae7e-a39419fe343f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dtb6s" Jul 29 20:29:28.085845 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:28.085691 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n57b9\" (UniqueName: \"kubernetes.io/projected/4c203790-6f9a-4aab-ae7e-a39419fe343f-kube-api-access-n57b9\") pod \"cert-manager-operator-controller-manager-55c444bf86-dtb6s\" (UID: \"4c203790-6f9a-4aab-ae7e-a39419fe343f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dtb6s" Jul 29 20:29:28.086025 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:28.086004 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c203790-6f9a-4aab-ae7e-a39419fe343f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-dtb6s\" (UID: \"4c203790-6f9a-4aab-ae7e-a39419fe343f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dtb6s" Jul 29 20:29:28.097690 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:28.097634 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n57b9\" (UniqueName: \"kubernetes.io/projected/4c203790-6f9a-4aab-ae7e-a39419fe343f-kube-api-access-n57b9\") pod \"cert-manager-operator-controller-manager-55c444bf86-dtb6s\" (UID: \"4c203790-6f9a-4aab-ae7e-a39419fe343f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dtb6s" Jul 29 20:29:28.146960 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:28.146936 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dtb6s" Jul 29 20:29:28.296438 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:28.296410 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dtb6s"] Jul 29 20:29:28.297794 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:29:28.297764 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c203790_6f9a_4aab_ae7e_a39419fe343f.slice/crio-c502c9f21111bbf07771098638af3146ae013d891dd2d118f6bbdc8ac7f13e04 WatchSource:0}: Error finding container c502c9f21111bbf07771098638af3146ae013d891dd2d118f6bbdc8ac7f13e04: Status 404 returned error can't find the container with id c502c9f21111bbf07771098638af3146ae013d891dd2d118f6bbdc8ac7f13e04 Jul 29 20:29:29.215872 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:29.215837 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dtb6s" event={"ID":"4c203790-6f9a-4aab-ae7e-a39419fe343f","Type":"ContainerStarted","Data":"c502c9f21111bbf07771098638af3146ae013d891dd2d118f6bbdc8ac7f13e04"} Jul 29 20:29:30.222293 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:30.222208 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dtb6s" event={"ID":"4c203790-6f9a-4aab-ae7e-a39419fe343f","Type":"ContainerStarted","Data":"b147d11879346dc5d7a41038b7117463a36db744ad82fe516d267bbd8b5b76df"} Jul 29 20:29:30.262240 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:30.262189 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dtb6s" podStartSLOduration=1.646707267 podStartE2EDuration="3.262173463s" totalImagesPullingTime="1.615466196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:29:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:29:30.261558355 +0000 UTC m=+265.776864579" watchObservedRunningTime="2026-07-29 20:29:30.262173463 +0000 UTC m=+265.777479685" Jul 29 20:29:31.831980 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:31.831945 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kvgcs"] Jul 29 20:29:31.834119 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:31.834102 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-kvgcs" Jul 29 20:29:31.836683 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:31.836654 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 29 20:29:31.836909 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:31.836893 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 29 20:29:31.837599 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:31.837581 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-npxwr\"" Jul 29 20:29:31.843594 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:31.843571 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kvgcs"] Jul 29 20:29:31.914547 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:31.914502 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgctp\" (UniqueName: \"kubernetes.io/projected/d72dce9d-aeed-4df9-8f6c-28c48b82da82-kube-api-access-vgctp\") pod \"cert-manager-webhook-6486ffd997-kvgcs\" (UID: \"d72dce9d-aeed-4df9-8f6c-28c48b82da82\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kvgcs" Jul 29 20:29:31.914723 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:31.914567 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d72dce9d-aeed-4df9-8f6c-28c48b82da82-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kvgcs\" (UID: \"d72dce9d-aeed-4df9-8f6c-28c48b82da82\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kvgcs" Jul 29 20:29:32.015159 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:32.015121 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d72dce9d-aeed-4df9-8f6c-28c48b82da82-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kvgcs\" (UID: \"d72dce9d-aeed-4df9-8f6c-28c48b82da82\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kvgcs" Jul 29 20:29:32.015337 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:32.015208 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgctp\" (UniqueName: \"kubernetes.io/projected/d72dce9d-aeed-4df9-8f6c-28c48b82da82-kube-api-access-vgctp\") pod \"cert-manager-webhook-6486ffd997-kvgcs\" (UID: \"d72dce9d-aeed-4df9-8f6c-28c48b82da82\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kvgcs" Jul 29 20:29:32.026659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:32.026633 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d72dce9d-aeed-4df9-8f6c-28c48b82da82-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kvgcs\" (UID: \"d72dce9d-aeed-4df9-8f6c-28c48b82da82\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kvgcs" Jul 29 20:29:32.028565 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:32.028538 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgctp\" (UniqueName: \"kubernetes.io/projected/d72dce9d-aeed-4df9-8f6c-28c48b82da82-kube-api-access-vgctp\") pod \"cert-manager-webhook-6486ffd997-kvgcs\" (UID: \"d72dce9d-aeed-4df9-8f6c-28c48b82da82\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kvgcs" Jul 29 20:29:32.150105 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:32.150004 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-kvgcs" Jul 29 20:29:32.276605 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:32.276580 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kvgcs"] Jul 29 20:29:32.278683 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:29:32.278656 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd72dce9d_aeed_4df9_8f6c_28c48b82da82.slice/crio-ad1ae1c0e93e89ee4d50d72cb985295f9cb5c47b6e95bda802c3aefcca8878e2 WatchSource:0}: Error finding container ad1ae1c0e93e89ee4d50d72cb985295f9cb5c47b6e95bda802c3aefcca8878e2: Status 404 returned error can't find the container with id ad1ae1c0e93e89ee4d50d72cb985295f9cb5c47b6e95bda802c3aefcca8878e2 Jul 29 20:29:33.241331 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:33.241285 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-kvgcs" event={"ID":"d72dce9d-aeed-4df9-8f6c-28c48b82da82","Type":"ContainerStarted","Data":"ad1ae1c0e93e89ee4d50d72cb985295f9cb5c47b6e95bda802c3aefcca8878e2"} Jul 29 20:29:36.256235 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:36.256201 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-kvgcs" event={"ID":"d72dce9d-aeed-4df9-8f6c-28c48b82da82","Type":"ContainerStarted","Data":"90063782dfdb0612e5f9c5549ddc59f56e8fab087887ec5cd1b72640072540da"} Jul 29 20:29:36.256692 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:36.256262 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-kvgcs" Jul 29 20:29:36.274304 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:36.274250 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-kvgcs" podStartSLOduration=2.117165808 podStartE2EDuration="5.274236625s" totalImagesPullingTime="3.157070817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:29:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:29:36.272495593 +0000 UTC m=+271.787801814" watchObservedRunningTime="2026-07-29 20:29:36.274236625 +0000 UTC m=+271.789542847" Jul 29 20:29:42.262314 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:42.262287 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-kvgcs" Jul 29 20:29:42.837899 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:42.837867 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-7zm5g"] Jul 29 20:29:42.840374 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:42.840360 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-7zm5g" Jul 29 20:29:42.842799 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:42.842782 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-hc6tr\"" Jul 29 20:29:42.851507 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:42.851483 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-7zm5g"] Jul 29 20:29:42.901143 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:42.901108 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfr8t\" (UniqueName: \"kubernetes.io/projected/e567ec88-ff8e-40c5-af50-772ea34a9695-kube-api-access-vfr8t\") pod \"cert-manager-55b4d954d5-7zm5g\" (UID: \"e567ec88-ff8e-40c5-af50-772ea34a9695\") " pod="cert-manager/cert-manager-55b4d954d5-7zm5g" Jul 29 20:29:42.901282 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:42.901146 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e567ec88-ff8e-40c5-af50-772ea34a9695-bound-sa-token\") pod \"cert-manager-55b4d954d5-7zm5g\" (UID: \"e567ec88-ff8e-40c5-af50-772ea34a9695\") " pod="cert-manager/cert-manager-55b4d954d5-7zm5g" Jul 29 20:29:43.002139 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:43.002103 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfr8t\" (UniqueName: \"kubernetes.io/projected/e567ec88-ff8e-40c5-af50-772ea34a9695-kube-api-access-vfr8t\") pod \"cert-manager-55b4d954d5-7zm5g\" (UID: \"e567ec88-ff8e-40c5-af50-772ea34a9695\") " pod="cert-manager/cert-manager-55b4d954d5-7zm5g" Jul 29 20:29:43.002139 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:43.002141 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e567ec88-ff8e-40c5-af50-772ea34a9695-bound-sa-token\") pod \"cert-manager-55b4d954d5-7zm5g\" (UID: \"e567ec88-ff8e-40c5-af50-772ea34a9695\") " pod="cert-manager/cert-manager-55b4d954d5-7zm5g" Jul 29 20:29:43.010787 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:43.010758 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e567ec88-ff8e-40c5-af50-772ea34a9695-bound-sa-token\") pod \"cert-manager-55b4d954d5-7zm5g\" (UID: \"e567ec88-ff8e-40c5-af50-772ea34a9695\") " pod="cert-manager/cert-manager-55b4d954d5-7zm5g" Jul 29 20:29:43.010949 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:43.010933 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfr8t\" (UniqueName: \"kubernetes.io/projected/e567ec88-ff8e-40c5-af50-772ea34a9695-kube-api-access-vfr8t\") pod \"cert-manager-55b4d954d5-7zm5g\" (UID: \"e567ec88-ff8e-40c5-af50-772ea34a9695\") " pod="cert-manager/cert-manager-55b4d954d5-7zm5g" Jul 29 20:29:43.148480 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:43.148397 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-7zm5g" Jul 29 20:29:43.269822 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:43.269797 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-7zm5g"] Jul 29 20:29:43.272142 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:29:43.272118 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode567ec88_ff8e_40c5_af50_772ea34a9695.slice/crio-d9c8b30b78370a91e4081b7f4e88f9becdee23f10138b09988a2609a1334ab2f WatchSource:0}: Error finding container d9c8b30b78370a91e4081b7f4e88f9becdee23f10138b09988a2609a1334ab2f: Status 404 returned error can't find the container with id d9c8b30b78370a91e4081b7f4e88f9becdee23f10138b09988a2609a1334ab2f Jul 29 20:29:43.286232 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:43.286203 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-7zm5g" event={"ID":"e567ec88-ff8e-40c5-af50-772ea34a9695","Type":"ContainerStarted","Data":"d9c8b30b78370a91e4081b7f4e88f9becdee23f10138b09988a2609a1334ab2f"} Jul 29 20:29:44.291526 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:44.291488 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-7zm5g" event={"ID":"e567ec88-ff8e-40c5-af50-772ea34a9695","Type":"ContainerStarted","Data":"09da97415a68ce77c62ec2b46f76a76434819cec01a23e3b30893a5153528f31"} Jul 29 20:29:44.308122 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:44.308080 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-7zm5g" podStartSLOduration=2.308065762 podStartE2EDuration="2.308065762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:29:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:29:44.307515392 +0000 UTC m=+279.822821618" watchObservedRunningTime="2026-07-29 20:29:44.308065762 +0000 UTC m=+279.823371975" Jul 29 20:29:55.637302 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:55.637269 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fl7mc"] Jul 29 20:29:55.639495 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:55.639478 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fl7mc" Jul 29 20:29:55.641911 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:55.641888 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 29 20:29:55.641997 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:55.641895 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-4wzdd\"" Jul 29 20:29:55.642904 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:55.642889 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 29 20:29:55.649956 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:55.649930 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fl7mc"] Jul 29 20:29:55.704407 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:55.704375 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2123db89-19f4-4b1a-b680-bf53295d25e7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fl7mc\" (UID: \"2123db89-19f4-4b1a-b680-bf53295d25e7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fl7mc" Jul 29 20:29:55.704594 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:55.704412 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv5xd\" (UniqueName: \"kubernetes.io/projected/2123db89-19f4-4b1a-b680-bf53295d25e7-kube-api-access-lv5xd\") pod \"openshift-lws-operator-fd8ccff4c-fl7mc\" (UID: \"2123db89-19f4-4b1a-b680-bf53295d25e7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fl7mc" Jul 29 20:29:55.805373 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:55.805336 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2123db89-19f4-4b1a-b680-bf53295d25e7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fl7mc\" (UID: \"2123db89-19f4-4b1a-b680-bf53295d25e7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fl7mc" Jul 29 20:29:55.805373 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:55.805378 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv5xd\" (UniqueName: \"kubernetes.io/projected/2123db89-19f4-4b1a-b680-bf53295d25e7-kube-api-access-lv5xd\") pod \"openshift-lws-operator-fd8ccff4c-fl7mc\" (UID: \"2123db89-19f4-4b1a-b680-bf53295d25e7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fl7mc" Jul 29 20:29:55.805782 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:55.805758 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2123db89-19f4-4b1a-b680-bf53295d25e7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fl7mc\" (UID: \"2123db89-19f4-4b1a-b680-bf53295d25e7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fl7mc" Jul 29 20:29:55.816816 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:55.816785 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv5xd\" (UniqueName: \"kubernetes.io/projected/2123db89-19f4-4b1a-b680-bf53295d25e7-kube-api-access-lv5xd\") pod \"openshift-lws-operator-fd8ccff4c-fl7mc\" (UID: \"2123db89-19f4-4b1a-b680-bf53295d25e7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fl7mc" Jul 29 20:29:55.948189 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:55.948086 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fl7mc" Jul 29 20:29:56.072515 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:56.072486 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fl7mc"] Jul 29 20:29:56.075417 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:29:56.075388 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2123db89_19f4_4b1a_b680_bf53295d25e7.slice/crio-24a12f9cc4688692c6626fce7b0c370175981e2e5beb2d49e4ce9509d6f47c73 WatchSource:0}: Error finding container 24a12f9cc4688692c6626fce7b0c370175981e2e5beb2d49e4ce9509d6f47c73: Status 404 returned error can't find the container with id 24a12f9cc4688692c6626fce7b0c370175981e2e5beb2d49e4ce9509d6f47c73 Jul 29 20:29:56.343174 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:56.343142 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fl7mc" event={"ID":"2123db89-19f4-4b1a-b680-bf53295d25e7","Type":"ContainerStarted","Data":"24a12f9cc4688692c6626fce7b0c370175981e2e5beb2d49e4ce9509d6f47c73"} Jul 29 20:29:59.359805 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:59.359763 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fl7mc" event={"ID":"2123db89-19f4-4b1a-b680-bf53295d25e7","Type":"ContainerStarted","Data":"e2d5e97f18ee389f39211e9a66d99dcd85e0a31b97b07f6d9a46946505fca8fd"} Jul 29 20:29:59.377418 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:29:59.377368 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fl7mc" podStartSLOduration=1.873792712 podStartE2EDuration="4.377351924s" totalImagesPullingTime="2.503559212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:29:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:29:59.376432236 +0000 UTC m=+294.891738461" watchObservedRunningTime="2026-07-29 20:29:59.377351924 +0000 UTC m=+294.892658147" Jul 29 20:30:04.833947 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:04.833912 2559 kubelet.go:1648] "Image garbage collection succeeded" Jul 29 20:30:06.092994 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.092954 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8"] Jul 29 20:30:06.095188 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.095172 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8" Jul 29 20:30:06.098670 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.098644 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 29 20:30:06.098797 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.098648 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-khxqm\"" Jul 29 20:30:06.098797 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.098648 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 29 20:30:06.098797 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.098746 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 29 20:30:06.111405 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.111380 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8"] Jul 29 20:30:06.201658 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.201619 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wktvb\" (UniqueName: \"kubernetes.io/projected/02057b30-6881-43c6-82cf-96d87bf24444-kube-api-access-wktvb\") pod \"lws-controller-manager-5c7786b84c-v5lc8\" (UID: \"02057b30-6881-43c6-82cf-96d87bf24444\") " pod="openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8" Jul 29 20:30:06.201835 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.201670 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/02057b30-6881-43c6-82cf-96d87bf24444-cert\") pod \"lws-controller-manager-5c7786b84c-v5lc8\" (UID: \"02057b30-6881-43c6-82cf-96d87bf24444\") " pod="openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8" Jul 29 20:30:06.201835 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.201724 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/02057b30-6881-43c6-82cf-96d87bf24444-metrics-cert\") pod \"lws-controller-manager-5c7786b84c-v5lc8\" (UID: \"02057b30-6881-43c6-82cf-96d87bf24444\") " pod="openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8" Jul 29 20:30:06.201835 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.201759 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/02057b30-6881-43c6-82cf-96d87bf24444-manager-config\") pod \"lws-controller-manager-5c7786b84c-v5lc8\" (UID: \"02057b30-6881-43c6-82cf-96d87bf24444\") " pod="openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8" Jul 29 20:30:06.302165 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.302125 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/02057b30-6881-43c6-82cf-96d87bf24444-manager-config\") pod \"lws-controller-manager-5c7786b84c-v5lc8\" (UID: \"02057b30-6881-43c6-82cf-96d87bf24444\") " pod="openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8" Jul 29 20:30:06.302165 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.302166 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wktvb\" (UniqueName: \"kubernetes.io/projected/02057b30-6881-43c6-82cf-96d87bf24444-kube-api-access-wktvb\") pod \"lws-controller-manager-5c7786b84c-v5lc8\" (UID: \"02057b30-6881-43c6-82cf-96d87bf24444\") " pod="openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8" Jul 29 20:30:06.302377 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.302205 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/02057b30-6881-43c6-82cf-96d87bf24444-cert\") pod \"lws-controller-manager-5c7786b84c-v5lc8\" (UID: \"02057b30-6881-43c6-82cf-96d87bf24444\") " pod="openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8" Jul 29 20:30:06.302377 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.302259 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/02057b30-6881-43c6-82cf-96d87bf24444-metrics-cert\") pod \"lws-controller-manager-5c7786b84c-v5lc8\" (UID: \"02057b30-6881-43c6-82cf-96d87bf24444\") " pod="openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8" Jul 29 20:30:06.302820 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.302796 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/02057b30-6881-43c6-82cf-96d87bf24444-manager-config\") pod \"lws-controller-manager-5c7786b84c-v5lc8\" (UID: \"02057b30-6881-43c6-82cf-96d87bf24444\") " pod="openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8" Jul 29 20:30:06.304556 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.304534 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/02057b30-6881-43c6-82cf-96d87bf24444-cert\") pod \"lws-controller-manager-5c7786b84c-v5lc8\" (UID: \"02057b30-6881-43c6-82cf-96d87bf24444\") " pod="openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8" Jul 29 20:30:06.304681 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.304663 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/02057b30-6881-43c6-82cf-96d87bf24444-metrics-cert\") pod \"lws-controller-manager-5c7786b84c-v5lc8\" (UID: \"02057b30-6881-43c6-82cf-96d87bf24444\") " pod="openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8" Jul 29 20:30:06.312969 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.312944 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wktvb\" (UniqueName: \"kubernetes.io/projected/02057b30-6881-43c6-82cf-96d87bf24444-kube-api-access-wktvb\") pod \"lws-controller-manager-5c7786b84c-v5lc8\" (UID: \"02057b30-6881-43c6-82cf-96d87bf24444\") " pod="openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8" Jul 29 20:30:06.404853 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.404773 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8" Jul 29 20:30:06.533636 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.533614 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8"] Jul 29 20:30:06.536157 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:30:06.536129 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02057b30_6881_43c6_82cf_96d87bf24444.slice/crio-6337dde15411ac71e36881c0fe8500519e2fb2b65147582f5d7ea22ada7d16d0 WatchSource:0}: Error finding container 6337dde15411ac71e36881c0fe8500519e2fb2b65147582f5d7ea22ada7d16d0: Status 404 returned error can't find the container with id 6337dde15411ac71e36881c0fe8500519e2fb2b65147582f5d7ea22ada7d16d0 Jul 29 20:30:06.538146 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:06.538128 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 20:30:07.393579 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:07.393546 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8" event={"ID":"02057b30-6881-43c6-82cf-96d87bf24444","Type":"ContainerStarted","Data":"6337dde15411ac71e36881c0fe8500519e2fb2b65147582f5d7ea22ada7d16d0"} Jul 29 20:30:11.411673 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:11.411636 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8" event={"ID":"02057b30-6881-43c6-82cf-96d87bf24444","Type":"ContainerStarted","Data":"50eb6cf8e10db6e3f0851840ead8a07e3f8925cc5f7a3ae151b7446197a82336"} Jul 29 20:30:11.412153 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:11.411709 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8" Jul 29 20:30:11.428043 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:11.427987 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8" podStartSLOduration=1.564926486 podStartE2EDuration="5.427973267s" totalImagesPullingTime="3.863046781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:30:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:30:11.427809259 +0000 UTC m=+306.943115495" watchObservedRunningTime="2026-07-29 20:30:11.427973267 +0000 UTC m=+306.943279490" Jul 29 20:30:17.855619 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:17.855580 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp"] Jul 29 20:30:17.858614 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:17.858594 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" Jul 29 20:30:17.862412 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:17.862389 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 29 20:30:17.863020 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:17.863000 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 29 20:30:17.863298 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:17.863283 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k4bvh\"" Jul 29 20:30:17.863353 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:17.863315 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 29 20:30:17.863399 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:17.863316 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 29 20:30:17.883115 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:17.883090 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp"] Jul 29 20:30:18.003390 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:18.003356 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2366750e-0e86-48ec-9b08-0a118c63572b-webhook-cert\") pod \"opendatahub-operator-controller-manager-74f7855d79-slmhp\" (UID: \"2366750e-0e86-48ec-9b08-0a118c63572b\") " pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" Jul 29 20:30:18.003582 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:18.003421 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2366750e-0e86-48ec-9b08-0a118c63572b-tmp\") pod \"opendatahub-operator-controller-manager-74f7855d79-slmhp\" (UID: \"2366750e-0e86-48ec-9b08-0a118c63572b\") " pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" Jul 29 20:30:18.003582 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:18.003497 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25sr2\" (UniqueName: \"kubernetes.io/projected/2366750e-0e86-48ec-9b08-0a118c63572b-kube-api-access-25sr2\") pod \"opendatahub-operator-controller-manager-74f7855d79-slmhp\" (UID: \"2366750e-0e86-48ec-9b08-0a118c63572b\") " pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" Jul 29 20:30:18.003582 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:18.003531 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2366750e-0e86-48ec-9b08-0a118c63572b-manifests\") pod \"opendatahub-operator-controller-manager-74f7855d79-slmhp\" (UID: \"2366750e-0e86-48ec-9b08-0a118c63572b\") " pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" Jul 29 20:30:18.003701 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:18.003591 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2366750e-0e86-48ec-9b08-0a118c63572b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74f7855d79-slmhp\" (UID: \"2366750e-0e86-48ec-9b08-0a118c63572b\") " pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" Jul 29 20:30:18.104607 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:18.104571 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2366750e-0e86-48ec-9b08-0a118c63572b-tmp\") pod \"opendatahub-operator-controller-manager-74f7855d79-slmhp\" (UID: \"2366750e-0e86-48ec-9b08-0a118c63572b\") " pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" Jul 29 20:30:18.104767 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:18.104622 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25sr2\" (UniqueName: \"kubernetes.io/projected/2366750e-0e86-48ec-9b08-0a118c63572b-kube-api-access-25sr2\") pod \"opendatahub-operator-controller-manager-74f7855d79-slmhp\" (UID: \"2366750e-0e86-48ec-9b08-0a118c63572b\") " pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" Jul 29 20:30:18.104767 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:18.104642 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2366750e-0e86-48ec-9b08-0a118c63572b-manifests\") pod \"opendatahub-operator-controller-manager-74f7855d79-slmhp\" (UID: \"2366750e-0e86-48ec-9b08-0a118c63572b\") " pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" Jul 29 20:30:18.104848 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:18.104816 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2366750e-0e86-48ec-9b08-0a118c63572b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74f7855d79-slmhp\" (UID: \"2366750e-0e86-48ec-9b08-0a118c63572b\") " pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" Jul 29 20:30:18.104903 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:18.104877 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2366750e-0e86-48ec-9b08-0a118c63572b-webhook-cert\") pod \"opendatahub-operator-controller-manager-74f7855d79-slmhp\" (UID: \"2366750e-0e86-48ec-9b08-0a118c63572b\") " pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" Jul 29 20:30:18.104985 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:18.104964 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2366750e-0e86-48ec-9b08-0a118c63572b-manifests\") pod \"opendatahub-operator-controller-manager-74f7855d79-slmhp\" (UID: \"2366750e-0e86-48ec-9b08-0a118c63572b\") " pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" Jul 29 20:30:18.106935 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:18.106868 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2366750e-0e86-48ec-9b08-0a118c63572b-tmp\") pod \"opendatahub-operator-controller-manager-74f7855d79-slmhp\" (UID: \"2366750e-0e86-48ec-9b08-0a118c63572b\") " pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" Jul 29 20:30:18.107086 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:18.107071 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2366750e-0e86-48ec-9b08-0a118c63572b-webhook-cert\") pod \"opendatahub-operator-controller-manager-74f7855d79-slmhp\" (UID: \"2366750e-0e86-48ec-9b08-0a118c63572b\") " pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" Jul 29 20:30:18.107163 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:18.107148 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2366750e-0e86-48ec-9b08-0a118c63572b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74f7855d79-slmhp\" (UID: \"2366750e-0e86-48ec-9b08-0a118c63572b\") " pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" Jul 29 20:30:18.113542 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:18.113519 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25sr2\" (UniqueName: \"kubernetes.io/projected/2366750e-0e86-48ec-9b08-0a118c63572b-kube-api-access-25sr2\") pod \"opendatahub-operator-controller-manager-74f7855d79-slmhp\" (UID: \"2366750e-0e86-48ec-9b08-0a118c63572b\") " pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" Jul 29 20:30:18.168096 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:18.168066 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" Jul 29 20:30:18.309115 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:18.309063 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp"] Jul 29 20:30:18.312036 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:30:18.312009 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2366750e_0e86_48ec_9b08_0a118c63572b.slice/crio-5c2cdc1035c7c893ceba2f12a965c444e84cd36f544a963eebe62c13b1ffe7d2 WatchSource:0}: Error finding container 5c2cdc1035c7c893ceba2f12a965c444e84cd36f544a963eebe62c13b1ffe7d2: Status 404 returned error can't find the container with id 5c2cdc1035c7c893ceba2f12a965c444e84cd36f544a963eebe62c13b1ffe7d2 Jul 29 20:30:18.442903 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:18.442816 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" event={"ID":"2366750e-0e86-48ec-9b08-0a118c63572b","Type":"ContainerStarted","Data":"5c2cdc1035c7c893ceba2f12a965c444e84cd36f544a963eebe62c13b1ffe7d2"} Jul 29 20:30:22.417483 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:22.417451 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5c7786b84c-v5lc8" Jul 29 20:30:22.464167 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:22.464133 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" event={"ID":"2366750e-0e86-48ec-9b08-0a118c63572b","Type":"ContainerStarted","Data":"6981e84804974c64878f6e3d1b7861f389a3555fac80499a9b5c911ee72bd7d1"} Jul 29 20:30:23.469972 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:23.469938 2559 generic.go:356] "Generic (PLEG): container finished" podID="2366750e-0e86-48ec-9b08-0a118c63572b" containerID="6981e84804974c64878f6e3d1b7861f389a3555fac80499a9b5c911ee72bd7d1" exitCode=0 Jul 29 20:30:23.470435 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:23.470022 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" event={"ID":"2366750e-0e86-48ec-9b08-0a118c63572b","Type":"ContainerDied","Data":"6981e84804974c64878f6e3d1b7861f389a3555fac80499a9b5c911ee72bd7d1"} Jul 29 20:30:24.476257 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:24.476221 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" event={"ID":"2366750e-0e86-48ec-9b08-0a118c63572b","Type":"ContainerStarted","Data":"0e6a95412ce388e4b63f5aa25d214323afe1358d2728e764c4e3a8dbebce6ef2"} Jul 29 20:30:24.476718 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:24.476301 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" Jul 29 20:30:24.500623 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:24.500578 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" podStartSLOduration=2.717534009 podStartE2EDuration="7.500565381s" totalImagesPullingTime="3.699010007s" totalInitContainerRuntime="1.084021365s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:30:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:30:24.499119152 +0000 UTC m=+320.014425394" watchObservedRunningTime="2026-07-29 20:30:24.500565381 +0000 UTC m=+320.015871605" Jul 29 20:30:35.482160 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:35.482131 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-74f7855d79-slmhp" Jul 29 20:30:45.902498 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:45.902468 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7"] Jul 29 20:30:45.909600 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:45.909577 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:45.911902 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:45.911876 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 29 20:30:45.912702 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:45.912684 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 29 20:30:45.912810 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:45.912785 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 29 20:30:45.913280 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:45.913258 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 29 20:30:45.913342 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:45.913265 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 29 20:30:45.913398 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:45.913347 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 29 20:30:45.913557 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:45.913543 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wjt8d\"" Jul 29 20:30:45.925471 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:45.925447 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7"] Jul 29 20:30:46.045622 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.045588 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0a9764a5-7986-4e6c-86e6-c90d9059bc61-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-cl4t7\" (UID: \"0a9764a5-7986-4e6c-86e6-c90d9059bc61\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.045807 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.045634 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0a9764a5-7986-4e6c-86e6-c90d9059bc61-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-cl4t7\" (UID: \"0a9764a5-7986-4e6c-86e6-c90d9059bc61\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.045807 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.045664 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a9764a5-7986-4e6c-86e6-c90d9059bc61-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-cl4t7\" (UID: \"0a9764a5-7986-4e6c-86e6-c90d9059bc61\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.045807 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.045689 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0a9764a5-7986-4e6c-86e6-c90d9059bc61-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-cl4t7\" (UID: \"0a9764a5-7986-4e6c-86e6-c90d9059bc61\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.045807 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.045740 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a9764a5-7986-4e6c-86e6-c90d9059bc61-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-cl4t7\" (UID: \"0a9764a5-7986-4e6c-86e6-c90d9059bc61\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.045960 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.045841 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0a9764a5-7986-4e6c-86e6-c90d9059bc61-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-cl4t7\" (UID: \"0a9764a5-7986-4e6c-86e6-c90d9059bc61\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.045960 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.045874 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvbkl\" (UniqueName: \"kubernetes.io/projected/0a9764a5-7986-4e6c-86e6-c90d9059bc61-kube-api-access-xvbkl\") pod \"istiod-openshift-gateway-5677ccb49b-cl4t7\" (UID: \"0a9764a5-7986-4e6c-86e6-c90d9059bc61\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.147149 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.147111 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a9764a5-7986-4e6c-86e6-c90d9059bc61-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-cl4t7\" (UID: \"0a9764a5-7986-4e6c-86e6-c90d9059bc61\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.147350 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.147183 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0a9764a5-7986-4e6c-86e6-c90d9059bc61-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-cl4t7\" (UID: \"0a9764a5-7986-4e6c-86e6-c90d9059bc61\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.147350 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.147205 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvbkl\" (UniqueName: \"kubernetes.io/projected/0a9764a5-7986-4e6c-86e6-c90d9059bc61-kube-api-access-xvbkl\") pod \"istiod-openshift-gateway-5677ccb49b-cl4t7\" (UID: \"0a9764a5-7986-4e6c-86e6-c90d9059bc61\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.147350 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.147222 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0a9764a5-7986-4e6c-86e6-c90d9059bc61-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-cl4t7\" (UID: \"0a9764a5-7986-4e6c-86e6-c90d9059bc61\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.147350 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.147248 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0a9764a5-7986-4e6c-86e6-c90d9059bc61-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-cl4t7\" (UID: \"0a9764a5-7986-4e6c-86e6-c90d9059bc61\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.147350 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.147268 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a9764a5-7986-4e6c-86e6-c90d9059bc61-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-cl4t7\" (UID: \"0a9764a5-7986-4e6c-86e6-c90d9059bc61\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.147610 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.147417 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0a9764a5-7986-4e6c-86e6-c90d9059bc61-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-cl4t7\" (UID: \"0a9764a5-7986-4e6c-86e6-c90d9059bc61\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.148348 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.148323 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0a9764a5-7986-4e6c-86e6-c90d9059bc61-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-cl4t7\" (UID: \"0a9764a5-7986-4e6c-86e6-c90d9059bc61\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.149550 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.149524 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a9764a5-7986-4e6c-86e6-c90d9059bc61-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-cl4t7\" (UID: \"0a9764a5-7986-4e6c-86e6-c90d9059bc61\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.149741 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.149721 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0a9764a5-7986-4e6c-86e6-c90d9059bc61-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-cl4t7\" (UID: \"0a9764a5-7986-4e6c-86e6-c90d9059bc61\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.150030 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.150011 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0a9764a5-7986-4e6c-86e6-c90d9059bc61-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-cl4t7\" (UID: \"0a9764a5-7986-4e6c-86e6-c90d9059bc61\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.150094 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.150026 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0a9764a5-7986-4e6c-86e6-c90d9059bc61-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-cl4t7\" (UID: \"0a9764a5-7986-4e6c-86e6-c90d9059bc61\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.154828 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.154766 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvbkl\" (UniqueName: \"kubernetes.io/projected/0a9764a5-7986-4e6c-86e6-c90d9059bc61-kube-api-access-xvbkl\") pod \"istiod-openshift-gateway-5677ccb49b-cl4t7\" (UID: \"0a9764a5-7986-4e6c-86e6-c90d9059bc61\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.154918 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.154859 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a9764a5-7986-4e6c-86e6-c90d9059bc61-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-cl4t7\" (UID: \"0a9764a5-7986-4e6c-86e6-c90d9059bc61\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.219027 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.218989 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:46.368030 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.367997 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7"] Jul 29 20:30:46.371204 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:30:46.371175 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a9764a5_7986_4e6c_86e6_c90d9059bc61.slice/crio-91618a5740b2adf736933974c17f344429c4450bd5dabd0ab4ee9c48524eb722 WatchSource:0}: Error finding container 91618a5740b2adf736933974c17f344429c4450bd5dabd0ab4ee9c48524eb722: Status 404 returned error can't find the container with id 91618a5740b2adf736933974c17f344429c4450bd5dabd0ab4ee9c48524eb722 Jul 29 20:30:46.581672 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:46.581639 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" event={"ID":"0a9764a5-7986-4e6c-86e6-c90d9059bc61","Type":"ContainerStarted","Data":"91618a5740b2adf736933974c17f344429c4450bd5dabd0ab4ee9c48524eb722"} Jul 29 20:30:48.789650 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:48.789606 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 20:30:48.789962 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:48.789681 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 20:30:49.596334 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:49.596300 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" event={"ID":"0a9764a5-7986-4e6c-86e6-c90d9059bc61","Type":"ContainerStarted","Data":"63df2fc46b78efe687ed47e7b724329bc23a91dec1542e79db637094ab03727e"} Jul 29 20:30:49.596524 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:49.596502 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:49.598164 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:49.598141 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" Jul 29 20:30:49.642262 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:30:49.642216 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-cl4t7" podStartSLOduration=2.225833583 podStartE2EDuration="4.64220159s" totalImagesPullingTime="2.416368007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:30:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:30:49.619356942 +0000 UTC m=+345.134663193" watchObservedRunningTime="2026-07-29 20:30:49.64220159 +0000 UTC m=+345.157507814" Jul 29 20:32:23.571851 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:23.571814 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9c6bfddc8-j88hw"] Jul 29 20:32:48.595682 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:48.595612 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-9c6bfddc8-j88hw" podUID="7a75e126-5067-462e-9a90-7f912831af8d" containerName="console" containerID="cri-o://b748eadc7a848d49e73fb6487e6d302fde976c5fcf14ee413ba12af8f9ec5566" gracePeriod=15 Jul 29 20:32:48.839792 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:48.839767 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:32:48.939954 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:48.939854 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a75e126-5067-462e-9a90-7f912831af8d-console-oauth-config\") pod \"7a75e126-5067-462e-9a90-7f912831af8d\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " Jul 29 20:32:48.939954 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:48.939901 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-oauth-serving-cert\") pod \"7a75e126-5067-462e-9a90-7f912831af8d\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " Jul 29 20:32:48.939954 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:48.939934 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-console-config\") pod \"7a75e126-5067-462e-9a90-7f912831af8d\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " Jul 29 20:32:48.940280 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:48.939961 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a75e126-5067-462e-9a90-7f912831af8d-console-serving-cert\") pod \"7a75e126-5067-462e-9a90-7f912831af8d\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " Jul 29 20:32:48.940280 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:48.939980 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cdjs\" (UniqueName: \"kubernetes.io/projected/7a75e126-5067-462e-9a90-7f912831af8d-kube-api-access-8cdjs\") pod \"7a75e126-5067-462e-9a90-7f912831af8d\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " Jul 29 20:32:48.940280 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:48.940009 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-trusted-ca-bundle\") pod \"7a75e126-5067-462e-9a90-7f912831af8d\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " Jul 29 20:32:48.940280 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:48.940122 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-service-ca\") pod \"7a75e126-5067-462e-9a90-7f912831af8d\" (UID: \"7a75e126-5067-462e-9a90-7f912831af8d\") " Jul 29 20:32:48.940904 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:48.940871 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7a75e126-5067-462e-9a90-7f912831af8d" (UID: "7a75e126-5067-462e-9a90-7f912831af8d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 20:32:48.941039 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:48.940876 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-console-config" (OuterVolumeSpecName: "console-config") pod "7a75e126-5067-462e-9a90-7f912831af8d" (UID: "7a75e126-5067-462e-9a90-7f912831af8d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 20:32:48.941039 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:48.940977 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-service-ca" (OuterVolumeSpecName: "service-ca") pod "7a75e126-5067-462e-9a90-7f912831af8d" (UID: "7a75e126-5067-462e-9a90-7f912831af8d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 20:32:48.941039 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:48.940998 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7a75e126-5067-462e-9a90-7f912831af8d" (UID: "7a75e126-5067-462e-9a90-7f912831af8d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 20:32:48.942278 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:48.942256 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a75e126-5067-462e-9a90-7f912831af8d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7a75e126-5067-462e-9a90-7f912831af8d" (UID: "7a75e126-5067-462e-9a90-7f912831af8d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 20:32:48.942574 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:48.942554 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a75e126-5067-462e-9a90-7f912831af8d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7a75e126-5067-462e-9a90-7f912831af8d" (UID: "7a75e126-5067-462e-9a90-7f912831af8d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 20:32:48.942574 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:48.942564 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a75e126-5067-462e-9a90-7f912831af8d-kube-api-access-8cdjs" (OuterVolumeSpecName: "kube-api-access-8cdjs") pod "7a75e126-5067-462e-9a90-7f912831af8d" (UID: "7a75e126-5067-462e-9a90-7f912831af8d"). InnerVolumeSpecName "kube-api-access-8cdjs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 20:32:49.041424 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:49.041395 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a75e126-5067-462e-9a90-7f912831af8d-console-serving-cert\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:32:49.041424 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:49.041423 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cdjs\" (UniqueName: \"kubernetes.io/projected/7a75e126-5067-462e-9a90-7f912831af8d-kube-api-access-8cdjs\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:32:49.041587 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:49.041433 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-trusted-ca-bundle\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:32:49.041587 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:49.041442 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-service-ca\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:32:49.041587 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:49.041452 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a75e126-5067-462e-9a90-7f912831af8d-console-oauth-config\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:32:49.041587 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:49.041460 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-oauth-serving-cert\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:32:49.041587 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:49.041468 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a75e126-5067-462e-9a90-7f912831af8d-console-config\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:32:49.152847 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:49.152810 2559 generic.go:356] "Generic (PLEG): container finished" podID="7a75e126-5067-462e-9a90-7f912831af8d" containerID="b748eadc7a848d49e73fb6487e6d302fde976c5fcf14ee413ba12af8f9ec5566" exitCode=0 Jul 29 20:32:49.153032 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:49.152876 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9c6bfddc8-j88hw" Jul 29 20:32:49.153032 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:49.152894 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9c6bfddc8-j88hw" event={"ID":"7a75e126-5067-462e-9a90-7f912831af8d","Type":"ContainerDied","Data":"b748eadc7a848d49e73fb6487e6d302fde976c5fcf14ee413ba12af8f9ec5566"} Jul 29 20:32:49.153032 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:49.152946 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9c6bfddc8-j88hw" event={"ID":"7a75e126-5067-462e-9a90-7f912831af8d","Type":"ContainerDied","Data":"cfc4c73a9cc56f97a94cd0a63c53963dbb725f0145e7000e786f9eda20dcc8b0"} Jul 29 20:32:49.153032 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:49.152974 2559 scope.go:117] "RemoveContainer" containerID="b748eadc7a848d49e73fb6487e6d302fde976c5fcf14ee413ba12af8f9ec5566" Jul 29 20:32:49.161789 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:49.161772 2559 scope.go:117] "RemoveContainer" containerID="b748eadc7a848d49e73fb6487e6d302fde976c5fcf14ee413ba12af8f9ec5566" Jul 29 20:32:49.162076 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:32:49.162039 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b748eadc7a848d49e73fb6487e6d302fde976c5fcf14ee413ba12af8f9ec5566\": container with ID starting with b748eadc7a848d49e73fb6487e6d302fde976c5fcf14ee413ba12af8f9ec5566 not found: ID does not exist" containerID="b748eadc7a848d49e73fb6487e6d302fde976c5fcf14ee413ba12af8f9ec5566" Jul 29 20:32:49.162123 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:49.162084 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b748eadc7a848d49e73fb6487e6d302fde976c5fcf14ee413ba12af8f9ec5566"} err="failed to get container status \"b748eadc7a848d49e73fb6487e6d302fde976c5fcf14ee413ba12af8f9ec5566\": rpc error: code = NotFound desc = could not find container \"b748eadc7a848d49e73fb6487e6d302fde976c5fcf14ee413ba12af8f9ec5566\": container with ID starting with b748eadc7a848d49e73fb6487e6d302fde976c5fcf14ee413ba12af8f9ec5566 not found: ID does not exist" Jul 29 20:32:49.169207 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:49.169183 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9c6bfddc8-j88hw"] Jul 29 20:32:49.170994 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:49.170976 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9c6bfddc8-j88hw"] Jul 29 20:32:51.025877 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:32:51.025842 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a75e126-5067-462e-9a90-7f912831af8d" path="/var/lib/kubelet/pods/7a75e126-5067-462e-9a90-7f912831af8d/volumes" Jul 29 20:33:28.289804 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:28.289761 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:33:28.290426 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:28.290255 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a75e126-5067-462e-9a90-7f912831af8d" containerName="console" Jul 29 20:33:28.290426 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:28.290277 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a75e126-5067-462e-9a90-7f912831af8d" containerName="console" Jul 29 20:33:28.290426 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:28.290375 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a75e126-5067-462e-9a90-7f912831af8d" containerName="console" Jul 29 20:33:28.293207 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:28.293185 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-j98nb" Jul 29 20:33:28.295496 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:28.295468 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 29 20:33:28.295496 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:28.295468 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-m9rcf\"" Jul 29 20:33:28.295667 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:28.295471 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 29 20:33:28.296286 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:28.296272 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 29 20:33:28.298617 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:28.298594 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b63ac234-7510-4273-bf41-933e21cea421-config-file\") pod \"limitador-limitador-54d6d988d5-j98nb\" (UID: \"b63ac234-7510-4273-bf41-933e21cea421\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-j98nb" Jul 29 20:33:28.298726 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:28.298638 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qqnw\" (UniqueName: \"kubernetes.io/projected/b63ac234-7510-4273-bf41-933e21cea421-kube-api-access-2qqnw\") pod \"limitador-limitador-54d6d988d5-j98nb\" (UID: \"b63ac234-7510-4273-bf41-933e21cea421\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-j98nb" Jul 29 20:33:28.300366 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:28.300161 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:33:28.330816 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:28.330774 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:33:28.399425 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:28.399383 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b63ac234-7510-4273-bf41-933e21cea421-config-file\") pod \"limitador-limitador-54d6d988d5-j98nb\" (UID: \"b63ac234-7510-4273-bf41-933e21cea421\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-j98nb" Jul 29 20:33:28.399588 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:28.399444 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qqnw\" (UniqueName: \"kubernetes.io/projected/b63ac234-7510-4273-bf41-933e21cea421-kube-api-access-2qqnw\") pod \"limitador-limitador-54d6d988d5-j98nb\" (UID: \"b63ac234-7510-4273-bf41-933e21cea421\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-j98nb" Jul 29 20:33:28.400091 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:28.400070 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b63ac234-7510-4273-bf41-933e21cea421-config-file\") pod \"limitador-limitador-54d6d988d5-j98nb\" (UID: \"b63ac234-7510-4273-bf41-933e21cea421\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-j98nb" Jul 29 20:33:28.406700 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:28.406677 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qqnw\" (UniqueName: \"kubernetes.io/projected/b63ac234-7510-4273-bf41-933e21cea421-kube-api-access-2qqnw\") pod \"limitador-limitador-54d6d988d5-j98nb\" (UID: \"b63ac234-7510-4273-bf41-933e21cea421\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-j98nb" Jul 29 20:33:28.604000 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:28.603903 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-j98nb" Jul 29 20:33:28.736714 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:28.736572 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:33:28.739199 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:33:28.739165 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb63ac234_7510_4273_bf41_933e21cea421.slice/crio-2c3478f2695a67841ec7703b47c342112f436e1f95f74a7093ebadcc2422c76e WatchSource:0}: Error finding container 2c3478f2695a67841ec7703b47c342112f436e1f95f74a7093ebadcc2422c76e: Status 404 returned error can't find the container with id 2c3478f2695a67841ec7703b47c342112f436e1f95f74a7093ebadcc2422c76e Jul 29 20:33:29.344815 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:29.344771 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-j98nb" event={"ID":"b63ac234-7510-4273-bf41-933e21cea421","Type":"ContainerStarted","Data":"2c3478f2695a67841ec7703b47c342112f436e1f95f74a7093ebadcc2422c76e"} Jul 29 20:33:31.357138 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:31.357102 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-j98nb" event={"ID":"b63ac234-7510-4273-bf41-933e21cea421","Type":"ContainerStarted","Data":"732e2367e37e425a2208592138b39e3c7904b8433702f3c112dd1b127bd90ef5"} Jul 29 20:33:31.357530 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:31.357201 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-j98nb" Jul 29 20:33:31.372573 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:31.372515 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-j98nb" podStartSLOduration=0.958972005 podStartE2EDuration="3.372494994s" totalImagesPullingTime="2.413522989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:33:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:33:31.370894923 +0000 UTC m=+506.886201147" watchObservedRunningTime="2026-07-29 20:33:31.372494994 +0000 UTC m=+506.887801219" Jul 29 20:33:42.363213 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:33:42.363173 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-j98nb" Jul 29 20:34:45.103611 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:34:45.103570 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:35:04.741222 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:04.741187 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:35:18.624571 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:18.624532 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:35:30.226536 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:30.226454 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:35:42.423828 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.423796 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj"] Jul 29 20:35:42.426912 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.426893 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:42.430306 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.430283 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 29 20:35:42.430306 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.430299 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bcvdz\"" Jul 29 20:35:42.430476 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.430286 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 29 20:35:42.430476 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.430286 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 29 20:35:42.436383 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.436360 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj"] Jul 29 20:35:42.531913 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.531880 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj\" (UID: \"82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:42.531913 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.531920 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n9zq\" (UniqueName: \"kubernetes.io/projected/82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada-kube-api-access-5n9zq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj\" (UID: \"82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:42.532147 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.531993 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj\" (UID: \"82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:42.532147 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.532038 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj\" (UID: \"82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:42.532147 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.532083 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj\" (UID: \"82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:42.532147 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.532135 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj\" (UID: \"82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:42.632908 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.632880 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5n9zq\" (UniqueName: \"kubernetes.io/projected/82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada-kube-api-access-5n9zq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj\" (UID: \"82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:42.633133 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.632969 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj\" (UID: \"82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:42.633133 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.633030 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj\" (UID: \"82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:42.633133 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.633086 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj\" (UID: \"82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:42.633133 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.633112 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj\" (UID: \"82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:42.633346 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.633144 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj\" (UID: \"82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:42.633451 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.633426 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj\" (UID: \"82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:42.633515 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.633454 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj\" (UID: \"82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:42.633515 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.633490 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj\" (UID: \"82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:42.635275 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.635255 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj\" (UID: \"82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:42.635476 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.635459 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj\" (UID: \"82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:42.639903 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.639887 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n9zq\" (UniqueName: \"kubernetes.io/projected/82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada-kube-api-access-5n9zq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj\" (UID: \"82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:42.737150 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.737068 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:42.862494 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.862471 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj"] Jul 29 20:35:42.864450 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:35:42.864419 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82dcc2f8_97a1_4f9c_8f69_aaf2ae947ada.slice/crio-e63df528e71d89fe645fca92d9d2a9a8f1cf2aec3a2eeb169e90da49b6fa4f99 WatchSource:0}: Error finding container e63df528e71d89fe645fca92d9d2a9a8f1cf2aec3a2eeb169e90da49b6fa4f99: Status 404 returned error can't find the container with id e63df528e71d89fe645fca92d9d2a9a8f1cf2aec3a2eeb169e90da49b6fa4f99 Jul 29 20:35:42.865701 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.865686 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 20:35:42.961501 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:42.961468 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" event={"ID":"82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada","Type":"ContainerStarted","Data":"e63df528e71d89fe645fca92d9d2a9a8f1cf2aec3a2eeb169e90da49b6fa4f99"} Jul 29 20:35:43.967698 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:43.967672 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" event={"ID":"82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada","Type":"ContainerStarted","Data":"bc5d31e80d844ca1d0aa218e718d64ddebbe1373ce203d2173389bf9cc5dd77c"} Jul 29 20:35:43.982899 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:43.982847 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" podStartSLOduration=0.965392356 podStartE2EDuration="1.982830917s" totalImagesPullingTime="1.017438561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:35:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:35:43.982476445 +0000 UTC m=+639.497782673" watchObservedRunningTime="2026-07-29 20:35:43.982830917 +0000 UTC m=+639.498137142" Jul 29 20:35:44.027834 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:44.027803 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:35:52.737596 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:52.737563 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:52.738226 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:52.737654 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:52.749600 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:52.749579 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:35:53.018868 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:35:53.018834 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj" Jul 29 20:36:05.535809 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:36:05.535773 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:36:21.929174 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:36:21.929138 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:37:53.995711 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:37:53.995677 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:38:03.653889 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:38:03.653850 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:38:13.443967 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:38:13.443932 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:38:22.744306 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:38:22.744275 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:40:03.639037 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:40:03.638957 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:40:13.123567 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:40:13.123535 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:41:01.126450 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:41:01.126391 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:41:11.130582 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:41:11.130542 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:41:37.130894 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:41:37.130806 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:41:45.132082 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:41:45.132034 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:41:55.224899 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:41:55.224862 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:42:04.629145 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:42:04.629106 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:42:15.428832 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:42:15.428795 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:42:25.727854 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:42:25.727818 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:42:47.425215 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:42:47.425178 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:42:58.926725 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:42:58.926645 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:43:20.938949 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:43:20.938915 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:43:33.631393 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:43:33.631357 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:43:47.034300 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:43:47.034271 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:43:57.725845 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:43:57.725809 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:44:14.834320 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:44:14.834280 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:44:23.932444 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:44:23.932410 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:44:33.433672 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:44:33.433594 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:44:44.130754 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:44:44.130721 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:44:52.642890 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:44:52.642857 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:45:02.427622 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:45:02.427589 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:45:13.930774 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:45:13.930741 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:45:22.944868 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:45:22.944834 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:47:34.722367 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:47:34.722285 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:47:45.026961 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:47:45.026926 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:48:04.128963 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:48:04.128927 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:48:26.128257 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:48:26.128218 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:48:36.622860 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:48:36.622826 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:48:46.536551 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:48:46.536517 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:48:57.322946 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:48:57.322865 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:49:08.237324 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:49:08.237283 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:49:17.684104 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:49:17.684068 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:49:27.934328 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:49:27.934295 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:49:39.326698 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:49:39.326661 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:49:45.683105 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:49:45.683067 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:50:07.138469 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:07.138428 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:50:17.335758 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:17.335727 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:50:30.333061 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:30.332962 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:50:43.534102 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.534066 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t"] Jul 29 20:50:43.537204 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.537182 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:43.539888 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.539865 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-d456fab9-kserve-self-signed-certs\"" Jul 29 20:50:43.550745 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.550724 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t"] Jul 29 20:50:43.599889 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.599863 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-home\") pod \"e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:43.600011 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.599918 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-dshm\") pod \"e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:43.600011 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.599973 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80141618-5889-4b0c-846e-64dd206347ca-tls-certs\") pod \"e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:43.600011 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.600005 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t694p\" (UniqueName: \"kubernetes.io/projected/80141618-5889-4b0c-846e-64dd206347ca-kube-api-access-t694p\") pod \"e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:43.600149 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.600081 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-model-cache\") pod \"e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:43.600195 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.600150 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-tmp-dir\") pod \"e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:43.700831 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.700807 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80141618-5889-4b0c-846e-64dd206347ca-tls-certs\") pod \"e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:43.700969 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.700840 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t694p\" (UniqueName: \"kubernetes.io/projected/80141618-5889-4b0c-846e-64dd206347ca-kube-api-access-t694p\") pod \"e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:43.701071 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.700966 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-model-cache\") pod \"e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:43.701071 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.701005 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-tmp-dir\") pod \"e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:43.701181 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.701079 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-home\") pod \"e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:43.701181 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.701131 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-dshm\") pod \"e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:43.701411 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.701385 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-model-cache\") pod \"e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:43.701411 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.701400 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-tmp-dir\") pod \"e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:43.701514 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.701499 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-home\") pod \"e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:43.703062 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.703034 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-dshm\") pod \"e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:43.703394 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.703379 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80141618-5889-4b0c-846e-64dd206347ca-tls-certs\") pod \"e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:43.708953 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.708932 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t694p\" (UniqueName: \"kubernetes.io/projected/80141618-5889-4b0c-846e-64dd206347ca-kube-api-access-t694p\") pod \"e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:43.846197 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.846116 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:43.974378 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:43.974357 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t"] Jul 29 20:50:43.976367 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:50:43.976329 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80141618_5889_4b0c_846e_64dd206347ca.slice/crio-4dc0688173aa72bdd8890d99328427fe0fa16516ab48aa2d3760f00039cc5275 WatchSource:0}: Error finding container 4dc0688173aa72bdd8890d99328427fe0fa16516ab48aa2d3760f00039cc5275: Status 404 returned error can't find the container with id 4dc0688173aa72bdd8890d99328427fe0fa16516ab48aa2d3760f00039cc5275 Jul 29 20:50:44.982393 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:44.982359 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" event={"ID":"80141618-5889-4b0c-846e-64dd206347ca","Type":"ContainerStarted","Data":"0b35b1dada9165c887acb3d4396a6cb28786a2251fc0ee0b2cf5573354a8cc81"} Jul 29 20:50:44.982869 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:44.982407 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" event={"ID":"80141618-5889-4b0c-846e-64dd206347ca","Type":"ContainerStarted","Data":"4dc0688173aa72bdd8890d99328427fe0fa16516ab48aa2d3760f00039cc5275"} Jul 29 20:50:45.000535 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:45.000492 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" podStartSLOduration=2.000476248 podStartE2EDuration="2.000476248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:50:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:50:44.999448312 +0000 UTC m=+1540.514754537" watchObservedRunningTime="2026-07-29 20:50:45.000476248 +0000 UTC m=+1540.515782472" Jul 29 20:50:47.431086 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:47.431040 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:50:53.847161 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:53.847123 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:53.847635 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:53.847176 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:53.859469 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:53.859443 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:50:54.033471 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:50:54.033442 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:51:11.433508 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:11.433477 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t"] Jul 29 20:51:14.626672 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:14.626642 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:51:26.033876 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:26.033840 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:51:26.453387 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:26.453285 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" podUID="80141618-5889-4b0c-846e-64dd206347ca" containerName="main" containerID="cri-o://0b35b1dada9165c887acb3d4396a6cb28786a2251fc0ee0b2cf5573354a8cc81" gracePeriod=45 Jul 29 20:51:26.707674 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:26.707620 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:51:26.870827 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:26.870797 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t694p\" (UniqueName: \"kubernetes.io/projected/80141618-5889-4b0c-846e-64dd206347ca-kube-api-access-t694p\") pod \"80141618-5889-4b0c-846e-64dd206347ca\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " Jul 29 20:51:26.870970 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:26.870838 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-tmp-dir\") pod \"80141618-5889-4b0c-846e-64dd206347ca\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " Jul 29 20:51:26.870970 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:26.870870 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-model-cache\") pod \"80141618-5889-4b0c-846e-64dd206347ca\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " Jul 29 20:51:26.870970 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:26.870901 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80141618-5889-4b0c-846e-64dd206347ca-tls-certs\") pod \"80141618-5889-4b0c-846e-64dd206347ca\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " Jul 29 20:51:26.870970 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:26.870936 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-home\") pod \"80141618-5889-4b0c-846e-64dd206347ca\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " Jul 29 20:51:26.871212 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:26.870986 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-dshm\") pod \"80141618-5889-4b0c-846e-64dd206347ca\" (UID: \"80141618-5889-4b0c-846e-64dd206347ca\") " Jul 29 20:51:26.871212 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:26.871131 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "80141618-5889-4b0c-846e-64dd206347ca" (UID: "80141618-5889-4b0c-846e-64dd206347ca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:51:26.871332 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:26.871204 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-model-cache" (OuterVolumeSpecName: "model-cache") pod "80141618-5889-4b0c-846e-64dd206347ca" (UID: "80141618-5889-4b0c-846e-64dd206347ca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:51:26.871332 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:26.871256 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-home" (OuterVolumeSpecName: "home") pod "80141618-5889-4b0c-846e-64dd206347ca" (UID: "80141618-5889-4b0c-846e-64dd206347ca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:51:26.871332 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:26.871314 2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-tmp-dir\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:51:26.871332 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:26.871327 2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-model-cache\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:51:26.871332 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:26.871336 2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-home\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:51:26.873036 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:26.873010 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80141618-5889-4b0c-846e-64dd206347ca-kube-api-access-t694p" (OuterVolumeSpecName: "kube-api-access-t694p") pod "80141618-5889-4b0c-846e-64dd206347ca" (UID: "80141618-5889-4b0c-846e-64dd206347ca"). InnerVolumeSpecName "kube-api-access-t694p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 20:51:26.873036 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:26.873012 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-dshm" (OuterVolumeSpecName: "dshm") pod "80141618-5889-4b0c-846e-64dd206347ca" (UID: "80141618-5889-4b0c-846e-64dd206347ca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 20:51:26.873188 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:26.873062 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80141618-5889-4b0c-846e-64dd206347ca-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "80141618-5889-4b0c-846e-64dd206347ca" (UID: "80141618-5889-4b0c-846e-64dd206347ca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 20:51:26.972074 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:26.972004 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t694p\" (UniqueName: \"kubernetes.io/projected/80141618-5889-4b0c-846e-64dd206347ca-kube-api-access-t694p\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:51:26.972074 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:26.972028 2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80141618-5889-4b0c-846e-64dd206347ca-tls-certs\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:51:26.972074 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:26.972043 2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80141618-5889-4b0c-846e-64dd206347ca-dshm\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:51:27.166505 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:27.166467 2559 generic.go:356] "Generic (PLEG): container finished" podID="80141618-5889-4b0c-846e-64dd206347ca" containerID="0b35b1dada9165c887acb3d4396a6cb28786a2251fc0ee0b2cf5573354a8cc81" exitCode=0 Jul 29 20:51:27.166880 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:27.166529 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" Jul 29 20:51:27.166880 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:27.166531 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" event={"ID":"80141618-5889-4b0c-846e-64dd206347ca","Type":"ContainerDied","Data":"0b35b1dada9165c887acb3d4396a6cb28786a2251fc0ee0b2cf5573354a8cc81"} Jul 29 20:51:27.166880 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:27.166575 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t" event={"ID":"80141618-5889-4b0c-846e-64dd206347ca","Type":"ContainerDied","Data":"4dc0688173aa72bdd8890d99328427fe0fa16516ab48aa2d3760f00039cc5275"} Jul 29 20:51:27.166880 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:27.166593 2559 scope.go:117] "RemoveContainer" containerID="0b35b1dada9165c887acb3d4396a6cb28786a2251fc0ee0b2cf5573354a8cc81" Jul 29 20:51:27.174761 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:27.174746 2559 scope.go:117] "RemoveContainer" containerID="0b35b1dada9165c887acb3d4396a6cb28786a2251fc0ee0b2cf5573354a8cc81" Jul 29 20:51:27.175010 ip-10-0-138-67 kubenswrapper[2559]: E0729 20:51:27.174989 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b35b1dada9165c887acb3d4396a6cb28786a2251fc0ee0b2cf5573354a8cc81\": container with ID starting with 0b35b1dada9165c887acb3d4396a6cb28786a2251fc0ee0b2cf5573354a8cc81 not found: ID does not exist" containerID="0b35b1dada9165c887acb3d4396a6cb28786a2251fc0ee0b2cf5573354a8cc81" Jul 29 20:51:27.175107 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:27.175015 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b35b1dada9165c887acb3d4396a6cb28786a2251fc0ee0b2cf5573354a8cc81"} err="failed to get container status \"0b35b1dada9165c887acb3d4396a6cb28786a2251fc0ee0b2cf5573354a8cc81\": rpc error: code = NotFound desc = could not find container \"0b35b1dada9165c887acb3d4396a6cb28786a2251fc0ee0b2cf5573354a8cc81\": container with ID starting with 0b35b1dada9165c887acb3d4396a6cb28786a2251fc0ee0b2cf5573354a8cc81 not found: ID does not exist" Jul 29 20:51:27.182789 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:27.182726 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t"] Jul 29 20:51:27.184685 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:27.184667 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-d456fab9-kserve-6597bc4655-q2s7t"] Jul 29 20:51:29.025170 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:29.025138 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80141618-5889-4b0c-846e-64dd206347ca" path="/var/lib/kubelet/pods/80141618-5889-4b0c-846e-64dd206347ca/volumes" Jul 29 20:51:33.436647 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:33.436608 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:51:44.022921 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:44.022881 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:51:58.926139 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:51:58.926040 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:52:14.341419 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:52:14.341381 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:52:24.732404 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:52:24.732367 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:52:34.831966 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:52:34.831928 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:52:43.334399 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:52:43.334365 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:52:52.432333 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:52:52.432288 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:53:06.427664 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:53:06.427632 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:53:15.143456 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:53:15.143422 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:53:24.228347 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:53:24.228310 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:53:36.027739 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:53:36.027653 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:53:48.526003 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:53:48.525969 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:53:56.229949 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:53:56.229913 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:54:19.027458 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:54:19.027425 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:54:30.829760 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:54:30.829720 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:54:37.026422 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:54:37.026393 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:54:50.929659 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:54:50.929624 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:55:00.229594 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:55:00.229509 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:55:13.234804 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:55:13.234774 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:55:27.633797 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:55:27.633761 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:55:37.428629 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:55:37.428592 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:55:47.528474 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:55:47.528441 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:55:56.233171 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:55:56.233134 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:56:15.987285 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:15.987250 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a64c614d-pthqr"] Jul 29 20:56:15.989545 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:15.987561 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80141618-5889-4b0c-846e-64dd206347ca" containerName="main" Jul 29 20:56:15.989545 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:15.987572 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="80141618-5889-4b0c-846e-64dd206347ca" containerName="main" Jul 29 20:56:15.989545 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:15.987623 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="80141618-5889-4b0c-846e-64dd206347ca" containerName="main" Jul 29 20:56:15.990419 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:15.990404 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a64c614d-pthqr" Jul 29 20:56:15.992586 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:15.992563 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wpqrc\"" Jul 29 20:56:15.992715 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:15.992618 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 29 20:56:16.004085 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:16.003948 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a64c614d-pthqr"] Jul 29 20:56:16.108329 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:16.108289 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0da1abbc-59fe-40ed-8239-85d587aa5ff3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a64c614d-pthqr\" (UID: \"0da1abbc-59fe-40ed-8239-85d587aa5ff3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a64c614d-pthqr" Jul 29 20:56:16.209416 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:16.209381 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0da1abbc-59fe-40ed-8239-85d587aa5ff3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a64c614d-pthqr\" (UID: \"0da1abbc-59fe-40ed-8239-85d587aa5ff3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a64c614d-pthqr" Jul 29 20:56:16.209977 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:16.209957 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0da1abbc-59fe-40ed-8239-85d587aa5ff3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a64c614d-pthqr\" (UID: \"0da1abbc-59fe-40ed-8239-85d587aa5ff3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a64c614d-pthqr" Jul 29 20:56:16.309275 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:16.309248 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a64c614d-pthqr" Jul 29 20:56:16.435570 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:16.435545 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a64c614d-pthqr"] Jul 29 20:56:16.437716 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:56:16.437690 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0da1abbc_59fe_40ed_8239_85d587aa5ff3.slice/crio-4bc9a0b37419bf4ae046c2d25eccd370cf56951c990b647e654217ff502b04fc WatchSource:0}: Error finding container 4bc9a0b37419bf4ae046c2d25eccd370cf56951c990b647e654217ff502b04fc: Status 404 returned error can't find the container with id 4bc9a0b37419bf4ae046c2d25eccd370cf56951c990b647e654217ff502b04fc Jul 29 20:56:16.439638 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:16.439621 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 20:56:16.453025 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:16.452996 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a64c614d-pthqr" event={"ID":"0da1abbc-59fe-40ed-8239-85d587aa5ff3","Type":"ContainerStarted","Data":"4bc9a0b37419bf4ae046c2d25eccd370cf56951c990b647e654217ff502b04fc"} Jul 29 20:56:19.469425 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:19.469393 2559 generic.go:356] "Generic (PLEG): container finished" podID="0da1abbc-59fe-40ed-8239-85d587aa5ff3" containerID="d3ae8dc6236d41929a617e589ddb5dc91b37892720b42f9d33ea4d26d2887c14" exitCode=0 Jul 29 20:56:19.469825 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:19.469491 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a64c614d-pthqr" event={"ID":"0da1abbc-59fe-40ed-8239-85d587aa5ff3","Type":"ContainerDied","Data":"d3ae8dc6236d41929a617e589ddb5dc91b37892720b42f9d33ea4d26d2887c14"} Jul 29 20:56:20.600243 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:20.600215 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a64c614d-pthqr" Jul 29 20:56:20.748130 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:20.748100 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0da1abbc-59fe-40ed-8239-85d587aa5ff3-maas-api-service-ca\") pod \"0da1abbc-59fe-40ed-8239-85d587aa5ff3\" (UID: \"0da1abbc-59fe-40ed-8239-85d587aa5ff3\") " Jul 29 20:56:20.748479 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:20.748451 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0da1abbc-59fe-40ed-8239-85d587aa5ff3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0da1abbc-59fe-40ed-8239-85d587aa5ff3" (UID: "0da1abbc-59fe-40ed-8239-85d587aa5ff3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 20:56:20.848746 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:20.848715 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0da1abbc-59fe-40ed-8239-85d587aa5ff3-maas-api-service-ca\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:56:21.479602 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:21.479523 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a64c614d-pthqr" Jul 29 20:56:21.479602 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:21.479557 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a64c614d-pthqr" event={"ID":"0da1abbc-59fe-40ed-8239-85d587aa5ff3","Type":"ContainerDied","Data":"4bc9a0b37419bf4ae046c2d25eccd370cf56951c990b647e654217ff502b04fc"} Jul 29 20:56:21.479778 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:21.479641 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4bc9a0b37419bf4ae046c2d25eccd370cf56951c990b647e654217ff502b04fc" Jul 29 20:56:31.235036 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:31.234945 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a64c614d-pthqr"] Jul 29 20:56:31.245112 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:31.245076 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a64c614d-pthqr"] Jul 29 20:56:33.025473 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:33.025439 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0da1abbc-59fe-40ed-8239-85d587aa5ff3" path="/var/lib/kubelet/pods/0da1abbc-59fe-40ed-8239-85d587aa5ff3/volumes" Jul 29 20:56:35.139783 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:35.139744 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:56:47.039269 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:56:47.039228 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:57:00.066144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:00.066105 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-eba524b8-wx2bq"] Jul 29 20:57:00.066541 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:00.066440 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0da1abbc-59fe-40ed-8239-85d587aa5ff3" containerName="revoke-keys" Jul 29 20:57:00.066541 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:00.066451 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0da1abbc-59fe-40ed-8239-85d587aa5ff3" containerName="revoke-keys" Jul 29 20:57:00.066541 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:00.066514 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0da1abbc-59fe-40ed-8239-85d587aa5ff3" containerName="revoke-keys" Jul 29 20:57:00.073603 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:00.073578 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-eba524b8-wx2bq" Jul 29 20:57:00.076746 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:00.076556 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 29 20:57:00.076746 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:00.076578 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wpqrc\"" Jul 29 20:57:00.077491 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:00.077465 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-eba524b8-wx2bq"] Jul 29 20:57:00.184762 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:00.184721 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/858ddaa6-71e7-470e-8122-604533f7ccfa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-eba524b8-wx2bq\" (UID: \"858ddaa6-71e7-470e-8122-604533f7ccfa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-eba524b8-wx2bq" Jul 29 20:57:00.285612 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:00.285572 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/858ddaa6-71e7-470e-8122-604533f7ccfa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-eba524b8-wx2bq\" (UID: \"858ddaa6-71e7-470e-8122-604533f7ccfa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-eba524b8-wx2bq" Jul 29 20:57:00.286237 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:00.286216 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/858ddaa6-71e7-470e-8122-604533f7ccfa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-eba524b8-wx2bq\" (UID: \"858ddaa6-71e7-470e-8122-604533f7ccfa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-eba524b8-wx2bq" Jul 29 20:57:00.384987 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:00.384883 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-eba524b8-wx2bq" Jul 29 20:57:00.519588 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:00.519555 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-eba524b8-wx2bq"] Jul 29 20:57:00.522836 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:57:00.522802 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod858ddaa6_71e7_470e_8122_604533f7ccfa.slice/crio-7fbf3c140ddaed705e50d963e431af7f525ff8e5dadbbd322b7156b08b90decc WatchSource:0}: Error finding container 7fbf3c140ddaed705e50d963e431af7f525ff8e5dadbbd322b7156b08b90decc: Status 404 returned error can't find the container with id 7fbf3c140ddaed705e50d963e431af7f525ff8e5dadbbd322b7156b08b90decc Jul 29 20:57:00.668931 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:00.668839 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-eba524b8-wx2bq" event={"ID":"858ddaa6-71e7-470e-8122-604533f7ccfa","Type":"ContainerStarted","Data":"7fbf3c140ddaed705e50d963e431af7f525ff8e5dadbbd322b7156b08b90decc"} Jul 29 20:57:01.675014 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:01.674980 2559 generic.go:356] "Generic (PLEG): container finished" podID="858ddaa6-71e7-470e-8122-604533f7ccfa" containerID="9b122da809f9a1618a9e42f1ce1f03871e243b63a8c94a952a4b69120fb4d2db" exitCode=0 Jul 29 20:57:01.675449 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:01.675089 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-eba524b8-wx2bq" event={"ID":"858ddaa6-71e7-470e-8122-604533f7ccfa","Type":"ContainerDied","Data":"9b122da809f9a1618a9e42f1ce1f03871e243b63a8c94a952a4b69120fb4d2db"} Jul 29 20:57:02.812390 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:02.812359 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-eba524b8-wx2bq" Jul 29 20:57:02.909024 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:02.908983 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/858ddaa6-71e7-470e-8122-604533f7ccfa-maas-api-service-ca\") pod \"858ddaa6-71e7-470e-8122-604533f7ccfa\" (UID: \"858ddaa6-71e7-470e-8122-604533f7ccfa\") " Jul 29 20:57:02.909388 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:02.909360 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/858ddaa6-71e7-470e-8122-604533f7ccfa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "858ddaa6-71e7-470e-8122-604533f7ccfa" (UID: "858ddaa6-71e7-470e-8122-604533f7ccfa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 20:57:03.009677 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:03.009647 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/858ddaa6-71e7-470e-8122-604533f7ccfa-maas-api-service-ca\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:57:03.685140 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:03.685103 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-eba524b8-wx2bq" event={"ID":"858ddaa6-71e7-470e-8122-604533f7ccfa","Type":"ContainerDied","Data":"7fbf3c140ddaed705e50d963e431af7f525ff8e5dadbbd322b7156b08b90decc"} Jul 29 20:57:03.685342 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:03.685161 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-eba524b8-wx2bq" Jul 29 20:57:03.685342 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:03.685176 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7fbf3c140ddaed705e50d963e431af7f525ff8e5dadbbd322b7156b08b90decc" Jul 29 20:57:06.428410 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:06.428374 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-eba524b8-wx2bq"] Jul 29 20:57:06.431355 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:06.431331 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-eba524b8-wx2bq"] Jul 29 20:57:07.024835 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:07.024804 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="858ddaa6-71e7-470e-8122-604533f7ccfa" path="/var/lib/kubelet/pods/858ddaa6-71e7-470e-8122-604533f7ccfa/volumes" Jul 29 20:57:10.541492 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:10.541456 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:57:22.530868 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:22.530835 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:57:35.805411 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:35.805366 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7cdd0d7d-7ntsj"] Jul 29 20:57:35.805921 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:35.805892 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="858ddaa6-71e7-470e-8122-604533f7ccfa" containerName="revoke-keys" Jul 29 20:57:35.805921 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:35.805912 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="858ddaa6-71e7-470e-8122-604533f7ccfa" containerName="revoke-keys" Jul 29 20:57:35.806073 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:35.806016 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="858ddaa6-71e7-470e-8122-604533f7ccfa" containerName="revoke-keys" Jul 29 20:57:35.809223 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:35.809202 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7cdd0d7d-7ntsj" Jul 29 20:57:35.811805 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:35.811780 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 29 20:57:35.811922 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:35.811781 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wpqrc\"" Jul 29 20:57:35.817262 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:35.817233 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7cdd0d7d-7ntsj"] Jul 29 20:57:35.897717 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:35.897685 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1151d218-78ea-458e-9d09-6d6bc3ceead6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7cdd0d7d-7ntsj\" (UID: \"1151d218-78ea-458e-9d09-6d6bc3ceead6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7cdd0d7d-7ntsj" Jul 29 20:57:35.998141 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:35.998112 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1151d218-78ea-458e-9d09-6d6bc3ceead6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7cdd0d7d-7ntsj\" (UID: \"1151d218-78ea-458e-9d09-6d6bc3ceead6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7cdd0d7d-7ntsj" Jul 29 20:57:35.998759 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:35.998735 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1151d218-78ea-458e-9d09-6d6bc3ceead6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7cdd0d7d-7ntsj\" (UID: \"1151d218-78ea-458e-9d09-6d6bc3ceead6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7cdd0d7d-7ntsj" Jul 29 20:57:36.120517 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:36.120423 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7cdd0d7d-7ntsj" Jul 29 20:57:36.248113 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:36.248087 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7cdd0d7d-7ntsj"] Jul 29 20:57:36.250365 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:57:36.250339 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1151d218_78ea_458e_9d09_6d6bc3ceead6.slice/crio-935c45a935048177a1a65591575ac85782c908013cb64ddd18ad59bc2c990d90 WatchSource:0}: Error finding container 935c45a935048177a1a65591575ac85782c908013cb64ddd18ad59bc2c990d90: Status 404 returned error can't find the container with id 935c45a935048177a1a65591575ac85782c908013cb64ddd18ad59bc2c990d90 Jul 29 20:57:36.840312 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:36.840285 2559 generic.go:356] "Generic (PLEG): container finished" podID="1151d218-78ea-458e-9d09-6d6bc3ceead6" containerID="dfab43a49d7e732950d98f7d719483fa309b7eaaf196dbb8c5d36b4a389f6271" exitCode=0 Jul 29 20:57:36.840634 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:36.840372 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7cdd0d7d-7ntsj" event={"ID":"1151d218-78ea-458e-9d09-6d6bc3ceead6","Type":"ContainerDied","Data":"dfab43a49d7e732950d98f7d719483fa309b7eaaf196dbb8c5d36b4a389f6271"} Jul 29 20:57:36.840634 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:36.840430 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7cdd0d7d-7ntsj" event={"ID":"1151d218-78ea-458e-9d09-6d6bc3ceead6","Type":"ContainerStarted","Data":"935c45a935048177a1a65591575ac85782c908013cb64ddd18ad59bc2c990d90"} Jul 29 20:57:37.974006 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:37.973980 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7cdd0d7d-7ntsj" Jul 29 20:57:38.015909 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:38.015880 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1151d218-78ea-458e-9d09-6d6bc3ceead6-maas-api-service-ca\") pod \"1151d218-78ea-458e-9d09-6d6bc3ceead6\" (UID: \"1151d218-78ea-458e-9d09-6d6bc3ceead6\") " Jul 29 20:57:38.016240 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:38.016219 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1151d218-78ea-458e-9d09-6d6bc3ceead6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1151d218-78ea-458e-9d09-6d6bc3ceead6" (UID: "1151d218-78ea-458e-9d09-6d6bc3ceead6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 20:57:38.117344 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:38.117279 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1151d218-78ea-458e-9d09-6d6bc3ceead6-maas-api-service-ca\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:57:38.851838 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:38.851807 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7cdd0d7d-7ntsj" Jul 29 20:57:38.852006 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:38.851810 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7cdd0d7d-7ntsj" event={"ID":"1151d218-78ea-458e-9d09-6d6bc3ceead6","Type":"ContainerDied","Data":"935c45a935048177a1a65591575ac85782c908013cb64ddd18ad59bc2c990d90"} Jul 29 20:57:38.852006 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:38.851935 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="935c45a935048177a1a65591575ac85782c908013cb64ddd18ad59bc2c990d90" Jul 29 20:57:40.734731 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:40.734698 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7cdd0d7d-7ntsj"] Jul 29 20:57:40.737576 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:40.737552 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7cdd0d7d-7ntsj"] Jul 29 20:57:41.025160 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:41.025124 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1151d218-78ea-458e-9d09-6d6bc3ceead6" path="/var/lib/kubelet/pods/1151d218-78ea-458e-9d09-6d6bc3ceead6/volumes" Jul 29 20:57:44.742391 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:44.742361 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:57:45.926699 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:45.926665 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc"] Jul 29 20:57:45.927104 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:45.926988 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1151d218-78ea-458e-9d09-6d6bc3ceead6" containerName="revoke-keys" Jul 29 20:57:45.927104 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:45.926998 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1151d218-78ea-458e-9d09-6d6bc3ceead6" containerName="revoke-keys" Jul 29 20:57:45.927104 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:45.927067 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1151d218-78ea-458e-9d09-6d6bc3ceead6" containerName="revoke-keys" Jul 29 20:57:45.930184 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:45.930165 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc" Jul 29 20:57:45.933116 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:45.933092 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-8e2d92b2\"" Jul 29 20:57:45.933237 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:45.933197 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-8e2d92b2-dockercfg-ssrtm\"" Jul 29 20:57:45.942917 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:45.942894 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc"] Jul 29 20:57:45.989967 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:45.989937 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353-plugins-config-volume\") pod \"payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc\" (UID: \"a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353\") " pod="openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc" Jul 29 20:57:45.990107 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:45.989984 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkkzw\" (UniqueName: \"kubernetes.io/projected/a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353-kube-api-access-xkkzw\") pod \"payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc\" (UID: \"a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353\") " pod="openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc" Jul 29 20:57:46.090719 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:46.090683 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353-plugins-config-volume\") pod \"payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc\" (UID: \"a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353\") " pod="openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc" Jul 29 20:57:46.090907 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:46.090740 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkkzw\" (UniqueName: \"kubernetes.io/projected/a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353-kube-api-access-xkkzw\") pod \"payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc\" (UID: \"a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353\") " pod="openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc" Jul 29 20:57:46.091406 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:46.091385 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353-plugins-config-volume\") pod \"payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc\" (UID: \"a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353\") " pod="openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc" Jul 29 20:57:46.098907 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:46.098884 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkkzw\" (UniqueName: \"kubernetes.io/projected/a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353-kube-api-access-xkkzw\") pod \"payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc\" (UID: \"a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353\") " pod="openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc" Jul 29 20:57:46.239459 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:46.239377 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc" Jul 29 20:57:46.367980 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:46.367953 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc"] Jul 29 20:57:46.370307 ip-10-0-138-67 kubenswrapper[2559]: W0729 20:57:46.370278 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0acb9c6_da70_40ac_b5bf_ed3e7fc6a353.slice/crio-327ecca85a5841417b346a5cc6181aee7cb4f9c880b418f1c96660c9cb5ffe70 WatchSource:0}: Error finding container 327ecca85a5841417b346a5cc6181aee7cb4f9c880b418f1c96660c9cb5ffe70: Status 404 returned error can't find the container with id 327ecca85a5841417b346a5cc6181aee7cb4f9c880b418f1c96660c9cb5ffe70 Jul 29 20:57:46.889361 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:46.889327 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc" event={"ID":"a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353","Type":"ContainerStarted","Data":"327ecca85a5841417b346a5cc6181aee7cb4f9c880b418f1c96660c9cb5ffe70"} Jul 29 20:57:48.902344 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:48.902292 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc" event={"ID":"a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353","Type":"ContainerStarted","Data":"e2db837443c85ec37076535fe3d52aee2f6baf9884440a72765a50c997ce7f22"} Jul 29 20:57:48.902745 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:48.902435 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc" Jul 29 20:57:48.918458 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:48.918415 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc" podStartSLOduration=2.3765981529999998 podStartE2EDuration="3.918399915s" totalImagesPullingTime="1.541801762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:57:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:57:48.917309751 +0000 UTC m=+1964.432615976" watchObservedRunningTime="2026-07-29 20:57:48.918399915 +0000 UTC m=+1964.433706139" Jul 29 20:57:57.218695 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:57.218610 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:57:59.908745 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:57:59.908717 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc" Jul 29 20:58:15.909071 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:58:15.909027 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc"] Jul 29 20:58:15.909539 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:58:15.909243 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc" podUID="a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353" containerName="payload-processing" containerID="cri-o://e2db837443c85ec37076535fe3d52aee2f6baf9884440a72765a50c997ce7f22" gracePeriod=30 Jul 29 20:58:16.037954 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:58:16.037922 2559 generic.go:356] "Generic (PLEG): container finished" podID="a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353" containerID="e2db837443c85ec37076535fe3d52aee2f6baf9884440a72765a50c997ce7f22" exitCode=0 Jul 29 20:58:16.038102 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:58:16.037999 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc" event={"ID":"a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353","Type":"ContainerDied","Data":"e2db837443c85ec37076535fe3d52aee2f6baf9884440a72765a50c997ce7f22"} Jul 29 20:58:16.184007 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:58:16.183730 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc" Jul 29 20:58:16.354708 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:58:16.354671 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkkzw\" (UniqueName: \"kubernetes.io/projected/a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353-kube-api-access-xkkzw\") pod \"a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353\" (UID: \"a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353\") " Jul 29 20:58:16.354708 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:58:16.354710 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353-plugins-config-volume\") pod \"a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353\" (UID: \"a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353\") " Jul 29 20:58:16.355144 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:58:16.355117 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353" (UID: "a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 20:58:16.356837 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:58:16.356814 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353-kube-api-access-xkkzw" (OuterVolumeSpecName: "kube-api-access-xkkzw") pod "a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353" (UID: "a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353"). InnerVolumeSpecName "kube-api-access-xkkzw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 20:58:16.455412 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:58:16.455333 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkkzw\" (UniqueName: \"kubernetes.io/projected/a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353-kube-api-access-xkkzw\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:58:16.455412 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:58:16.455360 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 20:58:17.043459 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:58:17.043430 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc" Jul 29 20:58:17.043850 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:58:17.043475 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc" event={"ID":"a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353","Type":"ContainerDied","Data":"327ecca85a5841417b346a5cc6181aee7cb4f9c880b418f1c96660c9cb5ffe70"} Jul 29 20:58:17.043850 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:58:17.043542 2559 scope.go:117] "RemoveContainer" containerID="e2db837443c85ec37076535fe3d52aee2f6baf9884440a72765a50c997ce7f22" Jul 29 20:58:17.066426 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:58:17.066401 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc"] Jul 29 20:58:17.075471 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:58:17.075446 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8e2d92b2-64684d8554-8fxlc"] Jul 29 20:58:19.024639 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:58:19.024606 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353" path="/var/lib/kubelet/pods/a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353/volumes" Jul 29 20:58:19.630165 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:58:19.630130 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 20:58:41.827537 ip-10-0-138-67 kubenswrapper[2559]: I0729 20:58:41.827506 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:00:12.355520 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.355485 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q"] Jul 29 21:00:12.355933 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.355799 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353" containerName="payload-processing" Jul 29 21:00:12.355933 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.355809 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353" containerName="payload-processing" Jul 29 21:00:12.355933 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.355877 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0acb9c6-da70-40ac-b5bf-ed3e7fc6a353" containerName="payload-processing" Jul 29 21:00:12.358781 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.358766 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q" Jul 29 21:00:12.361156 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.361130 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-4dbbbb61-dockercfg-9wzzg\"" Jul 29 21:00:12.361296 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.361197 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-4dbbbb61\"" Jul 29 21:00:12.368725 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.368702 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q"] Jul 29 21:00:12.385246 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.385222 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb"] Jul 29 21:00:12.388560 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.388544 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb" Jul 29 21:00:12.396092 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.396072 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb"] Jul 29 21:00:12.429354 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.429314 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f1787a4-8132-459c-bcd4-e87e0124c5db-plugins-config-volume\") pod \"payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q\" (UID: \"6f1787a4-8132-459c-bcd4-e87e0124c5db\") " pod="openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q" Jul 29 21:00:12.429500 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.429372 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70b5a2c6-76f8-454e-ada1-42f72fe7ec46-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb\" (UID: \"70b5a2c6-76f8-454e-ada1-42f72fe7ec46\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb" Jul 29 21:00:12.429500 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.429417 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v727\" (UniqueName: \"kubernetes.io/projected/6f1787a4-8132-459c-bcd4-e87e0124c5db-kube-api-access-6v727\") pod \"payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q\" (UID: \"6f1787a4-8132-459c-bcd4-e87e0124c5db\") " pod="openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q" Jul 29 21:00:12.429500 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.429493 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsv5p\" (UniqueName: \"kubernetes.io/projected/70b5a2c6-76f8-454e-ada1-42f72fe7ec46-kube-api-access-zsv5p\") pod \"payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb\" (UID: \"70b5a2c6-76f8-454e-ada1-42f72fe7ec46\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb" Jul 29 21:00:12.530688 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.530655 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v727\" (UniqueName: \"kubernetes.io/projected/6f1787a4-8132-459c-bcd4-e87e0124c5db-kube-api-access-6v727\") pod \"payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q\" (UID: \"6f1787a4-8132-459c-bcd4-e87e0124c5db\") " pod="openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q" Jul 29 21:00:12.530848 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.530705 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsv5p\" (UniqueName: \"kubernetes.io/projected/70b5a2c6-76f8-454e-ada1-42f72fe7ec46-kube-api-access-zsv5p\") pod \"payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb\" (UID: \"70b5a2c6-76f8-454e-ada1-42f72fe7ec46\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb" Jul 29 21:00:12.530848 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.530757 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f1787a4-8132-459c-bcd4-e87e0124c5db-plugins-config-volume\") pod \"payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q\" (UID: \"6f1787a4-8132-459c-bcd4-e87e0124c5db\") " pod="openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q" Jul 29 21:00:12.530848 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.530774 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70b5a2c6-76f8-454e-ada1-42f72fe7ec46-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb\" (UID: \"70b5a2c6-76f8-454e-ada1-42f72fe7ec46\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb" Jul 29 21:00:12.531475 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.531456 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f1787a4-8132-459c-bcd4-e87e0124c5db-plugins-config-volume\") pod \"payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q\" (UID: \"6f1787a4-8132-459c-bcd4-e87e0124c5db\") " pod="openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q" Jul 29 21:00:12.531537 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.531471 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70b5a2c6-76f8-454e-ada1-42f72fe7ec46-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb\" (UID: \"70b5a2c6-76f8-454e-ada1-42f72fe7ec46\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb" Jul 29 21:00:12.538062 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.538034 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v727\" (UniqueName: \"kubernetes.io/projected/6f1787a4-8132-459c-bcd4-e87e0124c5db-kube-api-access-6v727\") pod \"payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q\" (UID: \"6f1787a4-8132-459c-bcd4-e87e0124c5db\") " pod="openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q" Jul 29 21:00:12.538177 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.538104 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsv5p\" (UniqueName: \"kubernetes.io/projected/70b5a2c6-76f8-454e-ada1-42f72fe7ec46-kube-api-access-zsv5p\") pod \"payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb\" (UID: \"70b5a2c6-76f8-454e-ada1-42f72fe7ec46\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb" Jul 29 21:00:12.669084 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.669004 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q" Jul 29 21:00:12.699097 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.699068 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb" Jul 29 21:00:12.810747 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.810721 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q"] Jul 29 21:00:12.813605 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:00:12.813572 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f1787a4_8132_459c_bcd4_e87e0124c5db.slice/crio-5ccd4eb1f1a2ad89cb166ad89b9b456f84e88dddcc82b991c707e94501bcdf32 WatchSource:0}: Error finding container 5ccd4eb1f1a2ad89cb166ad89b9b456f84e88dddcc82b991c707e94501bcdf32: Status 404 returned error can't find the container with id 5ccd4eb1f1a2ad89cb166ad89b9b456f84e88dddcc82b991c707e94501bcdf32 Jul 29 21:00:12.845763 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:12.845735 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb"] Jul 29 21:00:12.847697 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:00:12.847667 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70b5a2c6_76f8_454e_ada1_42f72fe7ec46.slice/crio-5e65b8ffebf64b1d8581d3497b0bb57de00768deb4db23f923b89a94996cec5f WatchSource:0}: Error finding container 5e65b8ffebf64b1d8581d3497b0bb57de00768deb4db23f923b89a94996cec5f: Status 404 returned error can't find the container with id 5e65b8ffebf64b1d8581d3497b0bb57de00768deb4db23f923b89a94996cec5f Jul 29 21:00:13.580846 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:13.580814 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q" event={"ID":"6f1787a4-8132-459c-bcd4-e87e0124c5db","Type":"ContainerStarted","Data":"a097228cfa046b7435a7788658710327f2222b22212d6020c24b5c0bae79b222"} Jul 29 21:00:13.581301 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:13.580863 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q" event={"ID":"6f1787a4-8132-459c-bcd4-e87e0124c5db","Type":"ContainerStarted","Data":"5ccd4eb1f1a2ad89cb166ad89b9b456f84e88dddcc82b991c707e94501bcdf32"} Jul 29 21:00:13.581301 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:13.580912 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q" Jul 29 21:00:13.582457 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:13.582434 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb" event={"ID":"70b5a2c6-76f8-454e-ada1-42f72fe7ec46","Type":"ContainerStarted","Data":"0b61e966cb3012be25c4a6339a8c90138dd6e32990b543e995559267ba8102d8"} Jul 29 21:00:13.582576 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:13.582478 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb" Jul 29 21:00:13.582576 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:13.582494 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb" event={"ID":"70b5a2c6-76f8-454e-ada1-42f72fe7ec46","Type":"ContainerStarted","Data":"5e65b8ffebf64b1d8581d3497b0bb57de00768deb4db23f923b89a94996cec5f"} Jul 29 21:00:13.600914 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:13.600858 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q" podStartSLOduration=1.6008429199999998 podStartE2EDuration="1.60084292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:00:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:00:13.598120348 +0000 UTC m=+2109.113426572" watchObservedRunningTime="2026-07-29 21:00:13.60084292 +0000 UTC m=+2109.116149141" Jul 29 21:00:13.613618 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:13.613574 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb" podStartSLOduration=1.613559645 podStartE2EDuration="1.613559645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:00:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:00:13.611823847 +0000 UTC m=+2109.127130071" watchObservedRunningTime="2026-07-29 21:00:13.613559645 +0000 UTC m=+2109.128865860" Jul 29 21:00:15.322323 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:15.322286 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:00:24.589398 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:24.589369 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q" Jul 29 21:00:24.589851 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:24.589444 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb" Jul 29 21:00:52.694973 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:52.694938 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz"] Jul 29 21:00:52.698261 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:52.698243 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz" Jul 29 21:00:52.700839 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:52.700810 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-72dabfe4\"" Jul 29 21:00:52.700839 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:52.700815 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-72dabfe4-dockercfg-q7kc7\"" Jul 29 21:00:52.708248 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:52.708229 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz"] Jul 29 21:00:52.755255 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:52.755227 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7"] Jul 29 21:00:52.758433 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:52.758412 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7" Jul 29 21:00:52.770310 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:52.770281 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7"] Jul 29 21:00:52.785874 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:52.785836 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e31af4a8-9069-49e3-b7c5-b8c609596fcd-plugins-config-volume\") pod \"payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7\" (UID: \"e31af4a8-9069-49e3-b7c5-b8c609596fcd\") " pod="openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7" Jul 29 21:00:52.786081 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:52.786041 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2w5p\" (UniqueName: \"kubernetes.io/projected/e31af4a8-9069-49e3-b7c5-b8c609596fcd-kube-api-access-c2w5p\") pod \"payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7\" (UID: \"e31af4a8-9069-49e3-b7c5-b8c609596fcd\") " pod="openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7" Jul 29 21:00:52.887212 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:52.887180 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e31af4a8-9069-49e3-b7c5-b8c609596fcd-plugins-config-volume\") pod \"payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7\" (UID: \"e31af4a8-9069-49e3-b7c5-b8c609596fcd\") " pod="openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7" Jul 29 21:00:52.887212 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:52.887214 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rgkh\" (UniqueName: \"kubernetes.io/projected/3837e152-eecf-4986-8db4-bf2f5095ef76-kube-api-access-4rgkh\") pod \"payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz\" (UID: \"3837e152-eecf-4986-8db4-bf2f5095ef76\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz" Jul 29 21:00:52.887427 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:52.887247 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3837e152-eecf-4986-8db4-bf2f5095ef76-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz\" (UID: \"3837e152-eecf-4986-8db4-bf2f5095ef76\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz" Jul 29 21:00:52.887427 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:52.887364 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2w5p\" (UniqueName: \"kubernetes.io/projected/e31af4a8-9069-49e3-b7c5-b8c609596fcd-kube-api-access-c2w5p\") pod \"payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7\" (UID: \"e31af4a8-9069-49e3-b7c5-b8c609596fcd\") " pod="openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7" Jul 29 21:00:52.887926 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:52.887907 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e31af4a8-9069-49e3-b7c5-b8c609596fcd-plugins-config-volume\") pod \"payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7\" (UID: \"e31af4a8-9069-49e3-b7c5-b8c609596fcd\") " pod="openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7" Jul 29 21:00:52.895669 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:52.895636 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2w5p\" (UniqueName: \"kubernetes.io/projected/e31af4a8-9069-49e3-b7c5-b8c609596fcd-kube-api-access-c2w5p\") pod \"payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7\" (UID: \"e31af4a8-9069-49e3-b7c5-b8c609596fcd\") " pod="openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7" Jul 29 21:00:52.988466 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:52.988363 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rgkh\" (UniqueName: \"kubernetes.io/projected/3837e152-eecf-4986-8db4-bf2f5095ef76-kube-api-access-4rgkh\") pod \"payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz\" (UID: \"3837e152-eecf-4986-8db4-bf2f5095ef76\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz" Jul 29 21:00:52.988466 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:52.988426 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3837e152-eecf-4986-8db4-bf2f5095ef76-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz\" (UID: \"3837e152-eecf-4986-8db4-bf2f5095ef76\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz" Jul 29 21:00:52.989195 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:52.989173 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3837e152-eecf-4986-8db4-bf2f5095ef76-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz\" (UID: \"3837e152-eecf-4986-8db4-bf2f5095ef76\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz" Jul 29 21:00:52.996887 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:52.996851 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rgkh\" (UniqueName: \"kubernetes.io/projected/3837e152-eecf-4986-8db4-bf2f5095ef76-kube-api-access-4rgkh\") pod \"payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz\" (UID: \"3837e152-eecf-4986-8db4-bf2f5095ef76\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz" Jul 29 21:00:53.008979 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:53.008956 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz" Jul 29 21:00:53.068651 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:53.068599 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7" Jul 29 21:00:53.146841 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:53.146773 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz"] Jul 29 21:00:53.149119 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:00:53.149087 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3837e152_eecf_4986_8db4_bf2f5095ef76.slice/crio-c052975887f002993495096faa8201005509c610c1099d9b2623ab4dd469a127 WatchSource:0}: Error finding container c052975887f002993495096faa8201005509c610c1099d9b2623ab4dd469a127: Status 404 returned error can't find the container with id c052975887f002993495096faa8201005509c610c1099d9b2623ab4dd469a127 Jul 29 21:00:53.210623 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:53.210571 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7"] Jul 29 21:00:53.212961 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:00:53.212935 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode31af4a8_9069_49e3_b7c5_b8c609596fcd.slice/crio-d3a653bad73fc9ffc44a38ce8a33a2e38afe48d466b74335475b76d407890d71 WatchSource:0}: Error finding container d3a653bad73fc9ffc44a38ce8a33a2e38afe48d466b74335475b76d407890d71: Status 404 returned error can't find the container with id d3a653bad73fc9ffc44a38ce8a33a2e38afe48d466b74335475b76d407890d71 Jul 29 21:00:53.778389 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:53.778353 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7" event={"ID":"e31af4a8-9069-49e3-b7c5-b8c609596fcd","Type":"ContainerStarted","Data":"bd11b2e553893683623d6e6bb36dbdb3a670cfc1a8a3ed232ef55f014af3fa76"} Jul 29 21:00:53.778862 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:53.778533 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7" Jul 29 21:00:53.778862 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:53.778583 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7" event={"ID":"e31af4a8-9069-49e3-b7c5-b8c609596fcd","Type":"ContainerStarted","Data":"d3a653bad73fc9ffc44a38ce8a33a2e38afe48d466b74335475b76d407890d71"} Jul 29 21:00:53.785981 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:53.785941 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz" event={"ID":"3837e152-eecf-4986-8db4-bf2f5095ef76","Type":"ContainerStarted","Data":"f20a6ada6c7f0479703a6e0b2764f138ec5d9d58e070efa74476412235774826"} Jul 29 21:00:53.786161 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:53.786028 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz" Jul 29 21:00:53.786161 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:53.786070 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz" event={"ID":"3837e152-eecf-4986-8db4-bf2f5095ef76","Type":"ContainerStarted","Data":"c052975887f002993495096faa8201005509c610c1099d9b2623ab4dd469a127"} Jul 29 21:00:53.800166 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:53.800117 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7" podStartSLOduration=1.8001036209999999 podStartE2EDuration="1.800103621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:00:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:00:53.799358269 +0000 UTC m=+2149.314664494" watchObservedRunningTime="2026-07-29 21:00:53.800103621 +0000 UTC m=+2149.315409850" Jul 29 21:00:53.816652 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:53.816603 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz" podStartSLOduration=1.816589133 podStartE2EDuration="1.816589133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:00:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:00:53.815517729 +0000 UTC m=+2149.330823952" watchObservedRunningTime="2026-07-29 21:00:53.816589133 +0000 UTC m=+2149.331895354" Jul 29 21:00:59.627877 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:00:59.627777 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:01:04.792966 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:01:04.792933 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz" Jul 29 21:01:04.793431 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:01:04.793105 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7" Jul 29 21:03:05.073314 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:03:05.073240 2559 scope.go:117] "RemoveContainer" containerID="d3ae8dc6236d41929a617e589ddb5dc91b37892720b42f9d33ea4d26d2887c14" Jul 29 21:03:05.081235 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:03:05.081215 2559 scope.go:117] "RemoveContainer" containerID="9b122da809f9a1618a9e42f1ce1f03871e243b63a8c94a952a4b69120fb4d2db" Jul 29 21:04:05.108156 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:04:05.103538 2559 scope.go:117] "RemoveContainer" containerID="dfab43a49d7e732950d98f7d719483fa309b7eaaf196dbb8c5d36b4a389f6271" Jul 29 21:09:41.831244 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:41.831204 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:09:54.438801 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:54.438769 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7"] Jul 29 21:09:54.439298 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:54.438971 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7" podUID="e31af4a8-9069-49e3-b7c5-b8c609596fcd" containerName="payload-processing" containerID="cri-o://bd11b2e553893683623d6e6bb36dbdb3a670cfc1a8a3ed232ef55f014af3fa76" gracePeriod=30 Jul 29 21:09:54.451256 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:54.451228 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz"] Jul 29 21:09:54.451492 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:54.451449 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz" podUID="3837e152-eecf-4986-8db4-bf2f5095ef76" containerName="payload-pre-processing" containerID="cri-o://f20a6ada6c7f0479703a6e0b2764f138ec5d9d58e070efa74476412235774826" gracePeriod=30 Jul 29 21:09:54.702408 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:54.702346 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7" Jul 29 21:09:54.714070 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:54.714037 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2w5p\" (UniqueName: \"kubernetes.io/projected/e31af4a8-9069-49e3-b7c5-b8c609596fcd-kube-api-access-c2w5p\") pod \"e31af4a8-9069-49e3-b7c5-b8c609596fcd\" (UID: \"e31af4a8-9069-49e3-b7c5-b8c609596fcd\") " Jul 29 21:09:54.714161 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:54.714119 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e31af4a8-9069-49e3-b7c5-b8c609596fcd-plugins-config-volume\") pod \"e31af4a8-9069-49e3-b7c5-b8c609596fcd\" (UID: \"e31af4a8-9069-49e3-b7c5-b8c609596fcd\") " Jul 29 21:09:54.714459 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:54.714436 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e31af4a8-9069-49e3-b7c5-b8c609596fcd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e31af4a8-9069-49e3-b7c5-b8c609596fcd" (UID: "e31af4a8-9069-49e3-b7c5-b8c609596fcd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:09:54.715895 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:54.715875 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e31af4a8-9069-49e3-b7c5-b8c609596fcd-kube-api-access-c2w5p" (OuterVolumeSpecName: "kube-api-access-c2w5p") pod "e31af4a8-9069-49e3-b7c5-b8c609596fcd" (UID: "e31af4a8-9069-49e3-b7c5-b8c609596fcd"). InnerVolumeSpecName "kube-api-access-c2w5p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:09:54.726769 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:54.726745 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz" Jul 29 21:09:54.814922 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:54.814883 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3837e152-eecf-4986-8db4-bf2f5095ef76-plugins-config-volume\") pod \"3837e152-eecf-4986-8db4-bf2f5095ef76\" (UID: \"3837e152-eecf-4986-8db4-bf2f5095ef76\") " Jul 29 21:09:54.814922 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:54.814930 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rgkh\" (UniqueName: \"kubernetes.io/projected/3837e152-eecf-4986-8db4-bf2f5095ef76-kube-api-access-4rgkh\") pod \"3837e152-eecf-4986-8db4-bf2f5095ef76\" (UID: \"3837e152-eecf-4986-8db4-bf2f5095ef76\") " Jul 29 21:09:54.815182 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:54.815159 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2w5p\" (UniqueName: \"kubernetes.io/projected/e31af4a8-9069-49e3-b7c5-b8c609596fcd-kube-api-access-c2w5p\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:09:54.815221 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:54.815180 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e31af4a8-9069-49e3-b7c5-b8c609596fcd-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:09:54.815300 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:54.815278 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3837e152-eecf-4986-8db4-bf2f5095ef76-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3837e152-eecf-4986-8db4-bf2f5095ef76" (UID: "3837e152-eecf-4986-8db4-bf2f5095ef76"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:09:54.816961 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:54.816942 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3837e152-eecf-4986-8db4-bf2f5095ef76-kube-api-access-4rgkh" (OuterVolumeSpecName: "kube-api-access-4rgkh") pod "3837e152-eecf-4986-8db4-bf2f5095ef76" (UID: "3837e152-eecf-4986-8db4-bf2f5095ef76"). InnerVolumeSpecName "kube-api-access-4rgkh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:09:54.916301 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:54.916264 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3837e152-eecf-4986-8db4-bf2f5095ef76-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:09:54.916301 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:54.916301 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rgkh\" (UniqueName: \"kubernetes.io/projected/3837e152-eecf-4986-8db4-bf2f5095ef76-kube-api-access-4rgkh\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:09:55.328099 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:55.327994 2559 generic.go:356] "Generic (PLEG): container finished" podID="e31af4a8-9069-49e3-b7c5-b8c609596fcd" containerID="bd11b2e553893683623d6e6bb36dbdb3a670cfc1a8a3ed232ef55f014af3fa76" exitCode=0 Jul 29 21:09:55.328099 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:55.328039 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7" Jul 29 21:09:55.328099 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:55.328080 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7" event={"ID":"e31af4a8-9069-49e3-b7c5-b8c609596fcd","Type":"ContainerDied","Data":"bd11b2e553893683623d6e6bb36dbdb3a670cfc1a8a3ed232ef55f014af3fa76"} Jul 29 21:09:55.328337 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:55.328137 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7" event={"ID":"e31af4a8-9069-49e3-b7c5-b8c609596fcd","Type":"ContainerDied","Data":"d3a653bad73fc9ffc44a38ce8a33a2e38afe48d466b74335475b76d407890d71"} Jul 29 21:09:55.328337 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:55.328168 2559 scope.go:117] "RemoveContainer" containerID="bd11b2e553893683623d6e6bb36dbdb3a670cfc1a8a3ed232ef55f014af3fa76" Jul 29 21:09:55.329428 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:55.329398 2559 generic.go:356] "Generic (PLEG): container finished" podID="3837e152-eecf-4986-8db4-bf2f5095ef76" containerID="f20a6ada6c7f0479703a6e0b2764f138ec5d9d58e070efa74476412235774826" exitCode=0 Jul 29 21:09:55.329523 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:55.329437 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz" Jul 29 21:09:55.329523 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:55.329485 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz" event={"ID":"3837e152-eecf-4986-8db4-bf2f5095ef76","Type":"ContainerDied","Data":"f20a6ada6c7f0479703a6e0b2764f138ec5d9d58e070efa74476412235774826"} Jul 29 21:09:55.329590 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:55.329530 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz" event={"ID":"3837e152-eecf-4986-8db4-bf2f5095ef76","Type":"ContainerDied","Data":"c052975887f002993495096faa8201005509c610c1099d9b2623ab4dd469a127"} Jul 29 21:09:55.336854 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:55.336828 2559 scope.go:117] "RemoveContainer" containerID="bd11b2e553893683623d6e6bb36dbdb3a670cfc1a8a3ed232ef55f014af3fa76" Jul 29 21:09:55.337111 ip-10-0-138-67 kubenswrapper[2559]: E0729 21:09:55.337090 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd11b2e553893683623d6e6bb36dbdb3a670cfc1a8a3ed232ef55f014af3fa76\": container with ID starting with bd11b2e553893683623d6e6bb36dbdb3a670cfc1a8a3ed232ef55f014af3fa76 not found: ID does not exist" containerID="bd11b2e553893683623d6e6bb36dbdb3a670cfc1a8a3ed232ef55f014af3fa76" Jul 29 21:09:55.337177 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:55.337117 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd11b2e553893683623d6e6bb36dbdb3a670cfc1a8a3ed232ef55f014af3fa76"} err="failed to get container status \"bd11b2e553893683623d6e6bb36dbdb3a670cfc1a8a3ed232ef55f014af3fa76\": rpc error: code = NotFound desc = could not find container \"bd11b2e553893683623d6e6bb36dbdb3a670cfc1a8a3ed232ef55f014af3fa76\": container with ID starting with bd11b2e553893683623d6e6bb36dbdb3a670cfc1a8a3ed232ef55f014af3fa76 not found: ID does not exist" Jul 29 21:09:55.337177 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:55.337133 2559 scope.go:117] "RemoveContainer" containerID="f20a6ada6c7f0479703a6e0b2764f138ec5d9d58e070efa74476412235774826" Jul 29 21:09:55.344776 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:55.344757 2559 scope.go:117] "RemoveContainer" containerID="f20a6ada6c7f0479703a6e0b2764f138ec5d9d58e070efa74476412235774826" Jul 29 21:09:55.345152 ip-10-0-138-67 kubenswrapper[2559]: E0729 21:09:55.345124 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f20a6ada6c7f0479703a6e0b2764f138ec5d9d58e070efa74476412235774826\": container with ID starting with f20a6ada6c7f0479703a6e0b2764f138ec5d9d58e070efa74476412235774826 not found: ID does not exist" containerID="f20a6ada6c7f0479703a6e0b2764f138ec5d9d58e070efa74476412235774826" Jul 29 21:09:55.345270 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:55.345158 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f20a6ada6c7f0479703a6e0b2764f138ec5d9d58e070efa74476412235774826"} err="failed to get container status \"f20a6ada6c7f0479703a6e0b2764f138ec5d9d58e070efa74476412235774826\": rpc error: code = NotFound desc = could not find container \"f20a6ada6c7f0479703a6e0b2764f138ec5d9d58e070efa74476412235774826\": container with ID starting with f20a6ada6c7f0479703a6e0b2764f138ec5d9d58e070efa74476412235774826 not found: ID does not exist" Jul 29 21:09:55.347359 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:55.347337 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7"] Jul 29 21:09:55.350745 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:55.350715 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-72dabfe4-f9758fffd-2z2g7"] Jul 29 21:09:55.360285 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:55.360263 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz"] Jul 29 21:09:55.362596 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:55.362578 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-72dabfe4-587c78794d-dnwmz"] Jul 29 21:09:57.031785 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:57.031705 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3837e152-eecf-4986-8db4-bf2f5095ef76" path="/var/lib/kubelet/pods/3837e152-eecf-4986-8db4-bf2f5095ef76/volumes" Jul 29 21:09:57.032185 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:57.032039 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e31af4a8-9069-49e3-b7c5-b8c609596fcd" path="/var/lib/kubelet/pods/e31af4a8-9069-49e3-b7c5-b8c609596fcd/volumes" Jul 29 21:09:57.131960 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:09:57.131922 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:10:06.447729 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:06.447697 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt"] Jul 29 21:10:06.448171 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:06.448080 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e31af4a8-9069-49e3-b7c5-b8c609596fcd" containerName="payload-processing" Jul 29 21:10:06.448171 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:06.448093 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e31af4a8-9069-49e3-b7c5-b8c609596fcd" containerName="payload-processing" Jul 29 21:10:06.448171 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:06.448118 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3837e152-eecf-4986-8db4-bf2f5095ef76" containerName="payload-pre-processing" Jul 29 21:10:06.448171 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:06.448124 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3837e152-eecf-4986-8db4-bf2f5095ef76" containerName="payload-pre-processing" Jul 29 21:10:06.448306 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:06.448187 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e31af4a8-9069-49e3-b7c5-b8c609596fcd" containerName="payload-processing" Jul 29 21:10:06.448306 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:06.448198 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3837e152-eecf-4986-8db4-bf2f5095ef76" containerName="payload-pre-processing" Jul 29 21:10:06.452523 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:06.452507 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt" Jul 29 21:10:06.454839 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:06.454808 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-0ab55cec\"" Jul 29 21:10:06.455119 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:06.455105 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-0ab55cec-dockercfg-2c6t7\"" Jul 29 21:10:06.462432 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:06.462404 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt"] Jul 29 21:10:06.515791 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:06.515755 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1344d109-1384-4269-98da-0b74e9256ec9-plugins-config-volume\") pod \"payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt\" (UID: \"1344d109-1384-4269-98da-0b74e9256ec9\") " pod="openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt" Jul 29 21:10:06.515952 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:06.515813 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqrzh\" (UniqueName: \"kubernetes.io/projected/1344d109-1384-4269-98da-0b74e9256ec9-kube-api-access-sqrzh\") pod \"payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt\" (UID: \"1344d109-1384-4269-98da-0b74e9256ec9\") " pod="openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt" Jul 29 21:10:06.617129 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:06.617086 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqrzh\" (UniqueName: \"kubernetes.io/projected/1344d109-1384-4269-98da-0b74e9256ec9-kube-api-access-sqrzh\") pod \"payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt\" (UID: \"1344d109-1384-4269-98da-0b74e9256ec9\") " pod="openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt" Jul 29 21:10:06.617296 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:06.617221 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1344d109-1384-4269-98da-0b74e9256ec9-plugins-config-volume\") pod \"payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt\" (UID: \"1344d109-1384-4269-98da-0b74e9256ec9\") " pod="openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt" Jul 29 21:10:06.617925 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:06.617907 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1344d109-1384-4269-98da-0b74e9256ec9-plugins-config-volume\") pod \"payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt\" (UID: \"1344d109-1384-4269-98da-0b74e9256ec9\") " pod="openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt" Jul 29 21:10:06.628353 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:06.628322 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqrzh\" (UniqueName: \"kubernetes.io/projected/1344d109-1384-4269-98da-0b74e9256ec9-kube-api-access-sqrzh\") pod \"payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt\" (UID: \"1344d109-1384-4269-98da-0b74e9256ec9\") " pod="openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt" Jul 29 21:10:06.763987 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:06.763950 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt" Jul 29 21:10:06.909614 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:06.909586 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt"] Jul 29 21:10:06.912345 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:10:06.912314 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1344d109_1384_4269_98da_0b74e9256ec9.slice/crio-e65034735571fc77317630011231f62385c836054fe53d5433cf581156c5f9c8 WatchSource:0}: Error finding container e65034735571fc77317630011231f62385c836054fe53d5433cf581156c5f9c8: Status 404 returned error can't find the container with id e65034735571fc77317630011231f62385c836054fe53d5433cf581156c5f9c8 Jul 29 21:10:07.390473 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:07.390437 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt" event={"ID":"1344d109-1384-4269-98da-0b74e9256ec9","Type":"ContainerStarted","Data":"a8fe75c54ae6123b7d85a9595fdbd11bb1f5efd4cb2876fec434cace24b7b05a"} Jul 29 21:10:07.390643 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:07.390498 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt" event={"ID":"1344d109-1384-4269-98da-0b74e9256ec9","Type":"ContainerStarted","Data":"e65034735571fc77317630011231f62385c836054fe53d5433cf581156c5f9c8"} Jul 29 21:10:07.390643 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:07.390578 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt" Jul 29 21:10:07.415566 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:07.415494 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt" podStartSLOduration=1.415478685 podStartE2EDuration="1.415478685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:10:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:10:07.413978592 +0000 UTC m=+2702.929284818" watchObservedRunningTime="2026-07-29 21:10:07.415478685 +0000 UTC m=+2702.930784909" Jul 29 21:10:10.527319 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:10.527281 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:10:18.397253 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:18.397215 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt" Jul 29 21:10:27.398619 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:27.398584 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt"] Jul 29 21:10:27.399103 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:27.398807 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt" podUID="1344d109-1384-4269-98da-0b74e9256ec9" containerName="payload-processing" containerID="cri-o://a8fe75c54ae6123b7d85a9595fdbd11bb1f5efd4cb2876fec434cace24b7b05a" gracePeriod=30 Jul 29 21:10:27.648316 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:27.648288 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt" Jul 29 21:10:27.816359 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:27.816305 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1344d109-1384-4269-98da-0b74e9256ec9-plugins-config-volume\") pod \"1344d109-1384-4269-98da-0b74e9256ec9\" (UID: \"1344d109-1384-4269-98da-0b74e9256ec9\") " Jul 29 21:10:27.816359 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:27.816370 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqrzh\" (UniqueName: \"kubernetes.io/projected/1344d109-1384-4269-98da-0b74e9256ec9-kube-api-access-sqrzh\") pod \"1344d109-1384-4269-98da-0b74e9256ec9\" (UID: \"1344d109-1384-4269-98da-0b74e9256ec9\") " Jul 29 21:10:27.816793 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:27.816772 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1344d109-1384-4269-98da-0b74e9256ec9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1344d109-1384-4269-98da-0b74e9256ec9" (UID: "1344d109-1384-4269-98da-0b74e9256ec9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:10:27.818457 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:27.818424 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1344d109-1384-4269-98da-0b74e9256ec9-kube-api-access-sqrzh" (OuterVolumeSpecName: "kube-api-access-sqrzh") pod "1344d109-1384-4269-98da-0b74e9256ec9" (UID: "1344d109-1384-4269-98da-0b74e9256ec9"). InnerVolumeSpecName "kube-api-access-sqrzh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:10:27.917705 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:27.917666 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1344d109-1384-4269-98da-0b74e9256ec9-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:10:27.917705 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:27.917694 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqrzh\" (UniqueName: \"kubernetes.io/projected/1344d109-1384-4269-98da-0b74e9256ec9-kube-api-access-sqrzh\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:10:28.492851 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:28.492810 2559 generic.go:356] "Generic (PLEG): container finished" podID="1344d109-1384-4269-98da-0b74e9256ec9" containerID="a8fe75c54ae6123b7d85a9595fdbd11bb1f5efd4cb2876fec434cace24b7b05a" exitCode=0 Jul 29 21:10:28.493310 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:28.492858 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt" Jul 29 21:10:28.493310 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:28.492901 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt" event={"ID":"1344d109-1384-4269-98da-0b74e9256ec9","Type":"ContainerDied","Data":"a8fe75c54ae6123b7d85a9595fdbd11bb1f5efd4cb2876fec434cace24b7b05a"} Jul 29 21:10:28.493310 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:28.492946 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt" event={"ID":"1344d109-1384-4269-98da-0b74e9256ec9","Type":"ContainerDied","Data":"e65034735571fc77317630011231f62385c836054fe53d5433cf581156c5f9c8"} Jul 29 21:10:28.493310 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:28.492967 2559 scope.go:117] "RemoveContainer" containerID="a8fe75c54ae6123b7d85a9595fdbd11bb1f5efd4cb2876fec434cace24b7b05a" Jul 29 21:10:28.504310 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:28.504291 2559 scope.go:117] "RemoveContainer" containerID="a8fe75c54ae6123b7d85a9595fdbd11bb1f5efd4cb2876fec434cace24b7b05a" Jul 29 21:10:28.504585 ip-10-0-138-67 kubenswrapper[2559]: E0729 21:10:28.504564 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8fe75c54ae6123b7d85a9595fdbd11bb1f5efd4cb2876fec434cace24b7b05a\": container with ID starting with a8fe75c54ae6123b7d85a9595fdbd11bb1f5efd4cb2876fec434cace24b7b05a not found: ID does not exist" containerID="a8fe75c54ae6123b7d85a9595fdbd11bb1f5efd4cb2876fec434cace24b7b05a" Jul 29 21:10:28.504643 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:28.504593 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8fe75c54ae6123b7d85a9595fdbd11bb1f5efd4cb2876fec434cace24b7b05a"} err="failed to get container status \"a8fe75c54ae6123b7d85a9595fdbd11bb1f5efd4cb2876fec434cace24b7b05a\": rpc error: code = NotFound desc = could not find container \"a8fe75c54ae6123b7d85a9595fdbd11bb1f5efd4cb2876fec434cace24b7b05a\": container with ID starting with a8fe75c54ae6123b7d85a9595fdbd11bb1f5efd4cb2876fec434cace24b7b05a not found: ID does not exist" Jul 29 21:10:28.516653 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:28.516628 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt"] Jul 29 21:10:28.522563 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:28.522541 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0ab55cec-84d564b64b-fq7bt"] Jul 29 21:10:29.025184 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:29.025153 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1344d109-1384-4269-98da-0b74e9256ec9" path="/var/lib/kubelet/pods/1344d109-1384-4269-98da-0b74e9256ec9/volumes" Jul 29 21:10:30.122939 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:10:30.122904 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:11:09.431129 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:11:09.431094 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:13:03.489245 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:03.489203 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg"] Jul 29 21:13:03.489770 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:03.489753 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1344d109-1384-4269-98da-0b74e9256ec9" containerName="payload-processing" Jul 29 21:13:03.489843 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:03.489772 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1344d109-1384-4269-98da-0b74e9256ec9" containerName="payload-processing" Jul 29 21:13:03.489895 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:03.489848 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1344d109-1384-4269-98da-0b74e9256ec9" containerName="payload-processing" Jul 29 21:13:03.492857 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:03.492830 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg" Jul 29 21:13:03.495567 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:03.495546 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-6951c5fd-dockercfg-bqfp2\"" Jul 29 21:13:03.495697 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:03.495652 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-6951c5fd\"" Jul 29 21:13:03.504139 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:03.504106 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg"] Jul 29 21:13:03.648167 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:03.648133 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de5581db-28d4-454e-a6a3-3533df40ffac-plugins-config-volume\") pod \"payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg\" (UID: \"de5581db-28d4-454e-a6a3-3533df40ffac\") " pod="openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg" Jul 29 21:13:03.648362 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:03.648213 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn5xw\" (UniqueName: \"kubernetes.io/projected/de5581db-28d4-454e-a6a3-3533df40ffac-kube-api-access-rn5xw\") pod \"payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg\" (UID: \"de5581db-28d4-454e-a6a3-3533df40ffac\") " pod="openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg" Jul 29 21:13:03.749538 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:03.749503 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de5581db-28d4-454e-a6a3-3533df40ffac-plugins-config-volume\") pod \"payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg\" (UID: \"de5581db-28d4-454e-a6a3-3533df40ffac\") " pod="openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg" Jul 29 21:13:03.749735 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:03.749569 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn5xw\" (UniqueName: \"kubernetes.io/projected/de5581db-28d4-454e-a6a3-3533df40ffac-kube-api-access-rn5xw\") pod \"payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg\" (UID: \"de5581db-28d4-454e-a6a3-3533df40ffac\") " pod="openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg" Jul 29 21:13:03.750307 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:03.750281 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de5581db-28d4-454e-a6a3-3533df40ffac-plugins-config-volume\") pod \"payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg\" (UID: \"de5581db-28d4-454e-a6a3-3533df40ffac\") " pod="openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg" Jul 29 21:13:03.759397 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:03.759378 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn5xw\" (UniqueName: \"kubernetes.io/projected/de5581db-28d4-454e-a6a3-3533df40ffac-kube-api-access-rn5xw\") pod \"payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg\" (UID: \"de5581db-28d4-454e-a6a3-3533df40ffac\") " pod="openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg" Jul 29 21:13:03.804467 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:03.804418 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg" Jul 29 21:13:03.934256 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:03.934230 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg"] Jul 29 21:13:03.936424 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:13:03.936396 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde5581db_28d4_454e_a6a3_3533df40ffac.slice/crio-8e952f36c20d898da1a1bf0c3722d139709bff9f24138d0b1e7712c2507b028e WatchSource:0}: Error finding container 8e952f36c20d898da1a1bf0c3722d139709bff9f24138d0b1e7712c2507b028e: Status 404 returned error can't find the container with id 8e952f36c20d898da1a1bf0c3722d139709bff9f24138d0b1e7712c2507b028e Jul 29 21:13:04.219674 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:04.219583 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg" event={"ID":"de5581db-28d4-454e-a6a3-3533df40ffac","Type":"ContainerStarted","Data":"93a79f492d05dd110ae97fdf0c8d279edb3ed40254dea82b0fc2ad93d56cb65b"} Jul 29 21:13:04.219674 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:04.219645 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg" event={"ID":"de5581db-28d4-454e-a6a3-3533df40ffac","Type":"ContainerStarted","Data":"8e952f36c20d898da1a1bf0c3722d139709bff9f24138d0b1e7712c2507b028e"} Jul 29 21:13:04.219893 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:04.219714 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg" Jul 29 21:13:04.236005 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:04.235949 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg" podStartSLOduration=1.23593053 podStartE2EDuration="1.23593053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:13:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:13:04.234174447 +0000 UTC m=+2879.749480682" watchObservedRunningTime="2026-07-29 21:13:04.23593053 +0000 UTC m=+2879.751236756" Jul 29 21:13:06.834682 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:06.834645 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:13:15.226777 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:15.226746 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg" Jul 29 21:13:35.233378 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:35.233342 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:13:48.131952 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:13:48.131916 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:14:21.632388 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:21.632353 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:14:22.817425 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:22.817388 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg"] Jul 29 21:14:22.817884 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:22.817638 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg" podUID="de5581db-28d4-454e-a6a3-3533df40ffac" containerName="payload-processing" containerID="cri-o://93a79f492d05dd110ae97fdf0c8d279edb3ed40254dea82b0fc2ad93d56cb65b" gracePeriod=30 Jul 29 21:14:23.074083 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:23.074003 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg" Jul 29 21:14:23.125914 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:23.125883 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rn5xw\" (UniqueName: \"kubernetes.io/projected/de5581db-28d4-454e-a6a3-3533df40ffac-kube-api-access-rn5xw\") pod \"de5581db-28d4-454e-a6a3-3533df40ffac\" (UID: \"de5581db-28d4-454e-a6a3-3533df40ffac\") " Jul 29 21:14:23.126065 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:23.125980 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de5581db-28d4-454e-a6a3-3533df40ffac-plugins-config-volume\") pod \"de5581db-28d4-454e-a6a3-3533df40ffac\" (UID: \"de5581db-28d4-454e-a6a3-3533df40ffac\") " Jul 29 21:14:23.126324 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:23.126301 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de5581db-28d4-454e-a6a3-3533df40ffac-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "de5581db-28d4-454e-a6a3-3533df40ffac" (UID: "de5581db-28d4-454e-a6a3-3533df40ffac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:14:23.127905 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:23.127884 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de5581db-28d4-454e-a6a3-3533df40ffac-kube-api-access-rn5xw" (OuterVolumeSpecName: "kube-api-access-rn5xw") pod "de5581db-28d4-454e-a6a3-3533df40ffac" (UID: "de5581db-28d4-454e-a6a3-3533df40ffac"). InnerVolumeSpecName "kube-api-access-rn5xw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:14:23.226763 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:23.226739 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de5581db-28d4-454e-a6a3-3533df40ffac-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:14:23.226763 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:23.226763 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rn5xw\" (UniqueName: \"kubernetes.io/projected/de5581db-28d4-454e-a6a3-3533df40ffac-kube-api-access-rn5xw\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:14:23.581295 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:23.581259 2559 generic.go:356] "Generic (PLEG): container finished" podID="de5581db-28d4-454e-a6a3-3533df40ffac" containerID="93a79f492d05dd110ae97fdf0c8d279edb3ed40254dea82b0fc2ad93d56cb65b" exitCode=0 Jul 29 21:14:23.581481 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:23.581308 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg" Jul 29 21:14:23.581481 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:23.581350 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg" event={"ID":"de5581db-28d4-454e-a6a3-3533df40ffac","Type":"ContainerDied","Data":"93a79f492d05dd110ae97fdf0c8d279edb3ed40254dea82b0fc2ad93d56cb65b"} Jul 29 21:14:23.581481 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:23.581407 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg" event={"ID":"de5581db-28d4-454e-a6a3-3533df40ffac","Type":"ContainerDied","Data":"8e952f36c20d898da1a1bf0c3722d139709bff9f24138d0b1e7712c2507b028e"} Jul 29 21:14:23.581481 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:23.581430 2559 scope.go:117] "RemoveContainer" containerID="93a79f492d05dd110ae97fdf0c8d279edb3ed40254dea82b0fc2ad93d56cb65b" Jul 29 21:14:23.589991 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:23.589972 2559 scope.go:117] "RemoveContainer" containerID="93a79f492d05dd110ae97fdf0c8d279edb3ed40254dea82b0fc2ad93d56cb65b" Jul 29 21:14:23.590257 ip-10-0-138-67 kubenswrapper[2559]: E0729 21:14:23.590238 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93a79f492d05dd110ae97fdf0c8d279edb3ed40254dea82b0fc2ad93d56cb65b\": container with ID starting with 93a79f492d05dd110ae97fdf0c8d279edb3ed40254dea82b0fc2ad93d56cb65b not found: ID does not exist" containerID="93a79f492d05dd110ae97fdf0c8d279edb3ed40254dea82b0fc2ad93d56cb65b" Jul 29 21:14:23.590307 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:23.590264 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93a79f492d05dd110ae97fdf0c8d279edb3ed40254dea82b0fc2ad93d56cb65b"} err="failed to get container status \"93a79f492d05dd110ae97fdf0c8d279edb3ed40254dea82b0fc2ad93d56cb65b\": rpc error: code = NotFound desc = could not find container \"93a79f492d05dd110ae97fdf0c8d279edb3ed40254dea82b0fc2ad93d56cb65b\": container with ID starting with 93a79f492d05dd110ae97fdf0c8d279edb3ed40254dea82b0fc2ad93d56cb65b not found: ID does not exist" Jul 29 21:14:23.602375 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:23.602355 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg"] Jul 29 21:14:23.606804 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:23.606782 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6951c5fd-7f6c89d74b-rgmhg"] Jul 29 21:14:25.025667 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:25.025634 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de5581db-28d4-454e-a6a3-3533df40ffac" path="/var/lib/kubelet/pods/de5581db-28d4-454e-a6a3-3533df40ffac/volumes" Jul 29 21:14:30.630992 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:30.630916 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:14:38.514865 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.514832 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q"] Jul 29 21:14:38.515375 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.515109 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q" podUID="6f1787a4-8132-459c-bcd4-e87e0124c5db" containerName="payload-processing" containerID="cri-o://a097228cfa046b7435a7788658710327f2222b22212d6020c24b5c0bae79b222" gracePeriod=30 Jul 29 21:14:38.524613 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.524586 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb"] Jul 29 21:14:38.524851 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.524816 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb" podUID="70b5a2c6-76f8-454e-ada1-42f72fe7ec46" containerName="payload-pre-processing" containerID="cri-o://0b61e966cb3012be25c4a6339a8c90138dd6e32990b543e995559267ba8102d8" gracePeriod=30 Jul 29 21:14:38.653256 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.653221 2559 generic.go:356] "Generic (PLEG): container finished" podID="6f1787a4-8132-459c-bcd4-e87e0124c5db" containerID="a097228cfa046b7435a7788658710327f2222b22212d6020c24b5c0bae79b222" exitCode=0 Jul 29 21:14:38.653371 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.653304 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q" event={"ID":"6f1787a4-8132-459c-bcd4-e87e0124c5db","Type":"ContainerDied","Data":"a097228cfa046b7435a7788658710327f2222b22212d6020c24b5c0bae79b222"} Jul 29 21:14:38.654931 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.654905 2559 generic.go:356] "Generic (PLEG): container finished" podID="70b5a2c6-76f8-454e-ada1-42f72fe7ec46" containerID="0b61e966cb3012be25c4a6339a8c90138dd6e32990b543e995559267ba8102d8" exitCode=0 Jul 29 21:14:38.655031 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.654969 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb" event={"ID":"70b5a2c6-76f8-454e-ada1-42f72fe7ec46","Type":"ContainerDied","Data":"0b61e966cb3012be25c4a6339a8c90138dd6e32990b543e995559267ba8102d8"} Jul 29 21:14:38.794123 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.794096 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q" Jul 29 21:14:38.796824 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.796804 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb" Jul 29 21:14:38.850513 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.850485 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsv5p\" (UniqueName: \"kubernetes.io/projected/70b5a2c6-76f8-454e-ada1-42f72fe7ec46-kube-api-access-zsv5p\") pod \"70b5a2c6-76f8-454e-ada1-42f72fe7ec46\" (UID: \"70b5a2c6-76f8-454e-ada1-42f72fe7ec46\") " Jul 29 21:14:38.850513 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.850518 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f1787a4-8132-459c-bcd4-e87e0124c5db-plugins-config-volume\") pod \"6f1787a4-8132-459c-bcd4-e87e0124c5db\" (UID: \"6f1787a4-8132-459c-bcd4-e87e0124c5db\") " Jul 29 21:14:38.850800 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.850551 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v727\" (UniqueName: \"kubernetes.io/projected/6f1787a4-8132-459c-bcd4-e87e0124c5db-kube-api-access-6v727\") pod \"6f1787a4-8132-459c-bcd4-e87e0124c5db\" (UID: \"6f1787a4-8132-459c-bcd4-e87e0124c5db\") " Jul 29 21:14:38.850800 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.850572 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70b5a2c6-76f8-454e-ada1-42f72fe7ec46-plugins-config-volume\") pod \"70b5a2c6-76f8-454e-ada1-42f72fe7ec46\" (UID: \"70b5a2c6-76f8-454e-ada1-42f72fe7ec46\") " Jul 29 21:14:38.850958 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.850929 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70b5a2c6-76f8-454e-ada1-42f72fe7ec46-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "70b5a2c6-76f8-454e-ada1-42f72fe7ec46" (UID: "70b5a2c6-76f8-454e-ada1-42f72fe7ec46"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:14:38.851085 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.850938 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f1787a4-8132-459c-bcd4-e87e0124c5db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6f1787a4-8132-459c-bcd4-e87e0124c5db" (UID: "6f1787a4-8132-459c-bcd4-e87e0124c5db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:14:38.852756 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.852730 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f1787a4-8132-459c-bcd4-e87e0124c5db-kube-api-access-6v727" (OuterVolumeSpecName: "kube-api-access-6v727") pod "6f1787a4-8132-459c-bcd4-e87e0124c5db" (UID: "6f1787a4-8132-459c-bcd4-e87e0124c5db"). InnerVolumeSpecName "kube-api-access-6v727". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:14:38.852756 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.852744 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70b5a2c6-76f8-454e-ada1-42f72fe7ec46-kube-api-access-zsv5p" (OuterVolumeSpecName: "kube-api-access-zsv5p") pod "70b5a2c6-76f8-454e-ada1-42f72fe7ec46" (UID: "70b5a2c6-76f8-454e-ada1-42f72fe7ec46"). InnerVolumeSpecName "kube-api-access-zsv5p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:14:38.951974 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.951945 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v727\" (UniqueName: \"kubernetes.io/projected/6f1787a4-8132-459c-bcd4-e87e0124c5db-kube-api-access-6v727\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:14:38.951974 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.951971 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70b5a2c6-76f8-454e-ada1-42f72fe7ec46-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:14:38.951974 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.951981 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsv5p\" (UniqueName: \"kubernetes.io/projected/70b5a2c6-76f8-454e-ada1-42f72fe7ec46-kube-api-access-zsv5p\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:14:38.952213 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:38.951989 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f1787a4-8132-459c-bcd4-e87e0124c5db-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:14:39.661017 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:39.660991 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q" Jul 29 21:14:39.661017 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:39.660998 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q" event={"ID":"6f1787a4-8132-459c-bcd4-e87e0124c5db","Type":"ContainerDied","Data":"5ccd4eb1f1a2ad89cb166ad89b9b456f84e88dddcc82b991c707e94501bcdf32"} Jul 29 21:14:39.661520 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:39.661083 2559 scope.go:117] "RemoveContainer" containerID="a097228cfa046b7435a7788658710327f2222b22212d6020c24b5c0bae79b222" Jul 29 21:14:39.662426 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:39.662405 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb" event={"ID":"70b5a2c6-76f8-454e-ada1-42f72fe7ec46","Type":"ContainerDied","Data":"5e65b8ffebf64b1d8581d3497b0bb57de00768deb4db23f923b89a94996cec5f"} Jul 29 21:14:39.662514 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:39.662414 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb" Jul 29 21:14:39.669285 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:39.669271 2559 scope.go:117] "RemoveContainer" containerID="0b61e966cb3012be25c4a6339a8c90138dd6e32990b543e995559267ba8102d8" Jul 29 21:14:39.678144 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:39.678118 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb"] Jul 29 21:14:39.681970 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:39.681952 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4dbbbb61-c6974bc6d-fbrfb"] Jul 29 21:14:39.693712 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:39.693682 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q"] Jul 29 21:14:39.697426 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:39.697406 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4dbbbb61-5f4f77c8bb-bc48q"] Jul 29 21:14:41.024725 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:41.024695 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f1787a4-8132-459c-bcd4-e87e0124c5db" path="/var/lib/kubelet/pods/6f1787a4-8132-459c-bcd4-e87e0124c5db/volumes" Jul 29 21:14:41.025151 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:14:41.025065 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70b5a2c6-76f8-454e-ada1-42f72fe7ec46" path="/var/lib/kubelet/pods/70b5a2c6-76f8-454e-ada1-42f72fe7ec46/volumes" Jul 29 21:15:00.139851 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:00.139813 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755995-nxqjb"] Jul 29 21:15:00.142249 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:00.140143 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f1787a4-8132-459c-bcd4-e87e0124c5db" containerName="payload-processing" Jul 29 21:15:00.142249 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:00.140154 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f1787a4-8132-459c-bcd4-e87e0124c5db" containerName="payload-processing" Jul 29 21:15:00.142249 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:00.140172 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70b5a2c6-76f8-454e-ada1-42f72fe7ec46" containerName="payload-pre-processing" Jul 29 21:15:00.142249 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:00.140178 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="70b5a2c6-76f8-454e-ada1-42f72fe7ec46" containerName="payload-pre-processing" Jul 29 21:15:00.142249 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:00.140184 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de5581db-28d4-454e-a6a3-3533df40ffac" containerName="payload-processing" Jul 29 21:15:00.142249 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:00.140190 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="de5581db-28d4-454e-a6a3-3533df40ffac" containerName="payload-processing" Jul 29 21:15:00.142249 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:00.140251 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f1787a4-8132-459c-bcd4-e87e0124c5db" containerName="payload-processing" Jul 29 21:15:00.142249 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:00.140258 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="de5581db-28d4-454e-a6a3-3533df40ffac" containerName="payload-processing" Jul 29 21:15:00.142249 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:00.140266 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="70b5a2c6-76f8-454e-ada1-42f72fe7ec46" containerName="payload-pre-processing" Jul 29 21:15:00.143091 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:00.143076 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755995-nxqjb" Jul 29 21:15:00.145532 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:00.145512 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wpqrc\"" Jul 29 21:15:00.150469 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:00.150446 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755995-nxqjb"] Jul 29 21:15:00.153865 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:00.153846 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755995-nxqjb" Jul 29 21:15:00.282933 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:00.282906 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755995-nxqjb"] Jul 29 21:15:00.285378 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:15:00.285353 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f85175c_9e15_4a93_add1_9101f1897f37.slice/crio-5b8b5a69452e960263680a989335e4bcc7e6e16736335ffe61b67ebe84dcd528 WatchSource:0}: Error finding container 5b8b5a69452e960263680a989335e4bcc7e6e16736335ffe61b67ebe84dcd528: Status 404 returned error can't find the container with id 5b8b5a69452e960263680a989335e4bcc7e6e16736335ffe61b67ebe84dcd528 Jul 29 21:15:00.760252 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:00.760214 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755995-nxqjb" event={"ID":"5f85175c-9e15-4a93-add1-9101f1897f37","Type":"ContainerStarted","Data":"2d76dc71e4fab24ca3cde066d8c2524ef32f4d6f8e210db2023dfc3343a63580"} Jul 29 21:15:00.760435 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:00.760297 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755995-nxqjb" event={"ID":"5f85175c-9e15-4a93-add1-9101f1897f37","Type":"ContainerStarted","Data":"5b8b5a69452e960263680a989335e4bcc7e6e16736335ffe61b67ebe84dcd528"} Jul 29 21:15:00.776184 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:00.776090 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755995-nxqjb" podStartSLOduration=0.776075685 podStartE2EDuration="776.075685ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:15:00.775107193 +0000 UTC m=+2996.290413427" watchObservedRunningTime="2026-07-29 21:15:00.776075685 +0000 UTC m=+2996.291381909" Jul 29 21:15:01.766539 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:01.766505 2559 generic.go:356] "Generic (PLEG): container finished" podID="5f85175c-9e15-4a93-add1-9101f1897f37" containerID="2d76dc71e4fab24ca3cde066d8c2524ef32f4d6f8e210db2023dfc3343a63580" exitCode=0 Jul 29 21:15:01.767003 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:01.766597 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755995-nxqjb" event={"ID":"5f85175c-9e15-4a93-add1-9101f1897f37","Type":"ContainerDied","Data":"2d76dc71e4fab24ca3cde066d8c2524ef32f4d6f8e210db2023dfc3343a63580"} Jul 29 21:15:02.894890 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:02.894833 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755995-nxqjb" Jul 29 21:15:03.777918 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:03.777881 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755995-nxqjb" event={"ID":"5f85175c-9e15-4a93-add1-9101f1897f37","Type":"ContainerDied","Data":"5b8b5a69452e960263680a989335e4bcc7e6e16736335ffe61b67ebe84dcd528"} Jul 29 21:15:03.778111 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:03.777961 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b8b5a69452e960263680a989335e4bcc7e6e16736335ffe61b67ebe84dcd528" Jul 29 21:15:03.778111 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:03.777900 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755995-nxqjb" Jul 29 21:15:17.231721 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:17.231686 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:15:39.931478 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:39.931444 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:15:40.420276 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.420243 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g"] Jul 29 21:15:40.420582 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.420564 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f85175c-9e15-4a93-add1-9101f1897f37" containerName="cleanup" Jul 29 21:15:40.420582 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.420580 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f85175c-9e15-4a93-add1-9101f1897f37" containerName="cleanup" Jul 29 21:15:40.420770 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.420668 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f85175c-9e15-4a93-add1-9101f1897f37" containerName="cleanup" Jul 29 21:15:40.424727 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.424708 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g" Jul 29 21:15:40.427526 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.427506 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-fcb1aff7-dockercfg-xrfgh\"" Jul 29 21:15:40.427829 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.427802 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-fcb1aff7\"" Jul 29 21:15:40.435019 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.434993 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g"] Jul 29 21:15:40.469415 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.469388 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q"] Jul 29 21:15:40.472579 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.472561 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q" Jul 29 21:15:40.481778 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.481733 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q"] Jul 29 21:15:40.551086 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.551056 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd7jr\" (UniqueName: \"kubernetes.io/projected/64a7386b-6e4e-455b-a662-f80f2341c7d6-kube-api-access-bd7jr\") pod \"payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q\" (UID: \"64a7386b-6e4e-455b-a662-f80f2341c7d6\") " pod="openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q" Jul 29 21:15:40.551443 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.551134 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8vzj\" (UniqueName: \"kubernetes.io/projected/a2c62560-c0f9-48d7-a181-7af3b8ea562c-kube-api-access-t8vzj\") pod \"payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g\" (UID: \"a2c62560-c0f9-48d7-a181-7af3b8ea562c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g" Jul 29 21:15:40.551443 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.551172 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2c62560-c0f9-48d7-a181-7af3b8ea562c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g\" (UID: \"a2c62560-c0f9-48d7-a181-7af3b8ea562c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g" Jul 29 21:15:40.551443 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.551253 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64a7386b-6e4e-455b-a662-f80f2341c7d6-plugins-config-volume\") pod \"payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q\" (UID: \"64a7386b-6e4e-455b-a662-f80f2341c7d6\") " pod="openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q" Jul 29 21:15:40.652150 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.652119 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd7jr\" (UniqueName: \"kubernetes.io/projected/64a7386b-6e4e-455b-a662-f80f2341c7d6-kube-api-access-bd7jr\") pod \"payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q\" (UID: \"64a7386b-6e4e-455b-a662-f80f2341c7d6\") " pod="openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q" Jul 29 21:15:40.652340 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.652164 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8vzj\" (UniqueName: \"kubernetes.io/projected/a2c62560-c0f9-48d7-a181-7af3b8ea562c-kube-api-access-t8vzj\") pod \"payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g\" (UID: \"a2c62560-c0f9-48d7-a181-7af3b8ea562c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g" Jul 29 21:15:40.652340 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.652186 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2c62560-c0f9-48d7-a181-7af3b8ea562c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g\" (UID: \"a2c62560-c0f9-48d7-a181-7af3b8ea562c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g" Jul 29 21:15:40.652340 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.652242 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64a7386b-6e4e-455b-a662-f80f2341c7d6-plugins-config-volume\") pod \"payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q\" (UID: \"64a7386b-6e4e-455b-a662-f80f2341c7d6\") " pod="openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q" Jul 29 21:15:40.652968 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.652944 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64a7386b-6e4e-455b-a662-f80f2341c7d6-plugins-config-volume\") pod \"payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q\" (UID: \"64a7386b-6e4e-455b-a662-f80f2341c7d6\") " pod="openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q" Jul 29 21:15:40.653089 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.652952 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2c62560-c0f9-48d7-a181-7af3b8ea562c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g\" (UID: \"a2c62560-c0f9-48d7-a181-7af3b8ea562c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g" Jul 29 21:15:40.664631 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.664527 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8vzj\" (UniqueName: \"kubernetes.io/projected/a2c62560-c0f9-48d7-a181-7af3b8ea562c-kube-api-access-t8vzj\") pod \"payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g\" (UID: \"a2c62560-c0f9-48d7-a181-7af3b8ea562c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g" Jul 29 21:15:40.664631 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.664531 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd7jr\" (UniqueName: \"kubernetes.io/projected/64a7386b-6e4e-455b-a662-f80f2341c7d6-kube-api-access-bd7jr\") pod \"payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q\" (UID: \"64a7386b-6e4e-455b-a662-f80f2341c7d6\") " pod="openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q" Jul 29 21:15:40.737600 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.737529 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g" Jul 29 21:15:40.783105 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.783075 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q" Jul 29 21:15:40.877765 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.877741 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g"] Jul 29 21:15:40.881406 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:15:40.881377 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2c62560_c0f9_48d7_a181_7af3b8ea562c.slice/crio-0a532bb9568921712491d36acd2f7bf00744a12084f0c94086c77ec6d056585a WatchSource:0}: Error finding container 0a532bb9568921712491d36acd2f7bf00744a12084f0c94086c77ec6d056585a: Status 404 returned error can't find the container with id 0a532bb9568921712491d36acd2f7bf00744a12084f0c94086c77ec6d056585a Jul 29 21:15:40.929966 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.929937 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q"] Jul 29 21:15:40.935133 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:15:40.935110 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64a7386b_6e4e_455b_a662_f80f2341c7d6.slice/crio-4168dcdde1e46a99e07042dd7899ac21a23c6951f8135b0ba080a78db427a03d WatchSource:0}: Error finding container 4168dcdde1e46a99e07042dd7899ac21a23c6951f8135b0ba080a78db427a03d: Status 404 returned error can't find the container with id 4168dcdde1e46a99e07042dd7899ac21a23c6951f8135b0ba080a78db427a03d Jul 29 21:15:40.943733 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.943705 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q" event={"ID":"64a7386b-6e4e-455b-a662-f80f2341c7d6","Type":"ContainerStarted","Data":"4168dcdde1e46a99e07042dd7899ac21a23c6951f8135b0ba080a78db427a03d"} Jul 29 21:15:40.945339 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:40.945315 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g" event={"ID":"a2c62560-c0f9-48d7-a181-7af3b8ea562c","Type":"ContainerStarted","Data":"0a532bb9568921712491d36acd2f7bf00744a12084f0c94086c77ec6d056585a"} Jul 29 21:15:41.952154 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:41.952117 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q" event={"ID":"64a7386b-6e4e-455b-a662-f80f2341c7d6","Type":"ContainerStarted","Data":"e59dab8de08810759da87fa98fa373204914b033aba282fcaae69a770a60cd7a"} Jul 29 21:15:41.952640 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:41.952218 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q" Jul 29 21:15:41.953610 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:41.953585 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g" event={"ID":"a2c62560-c0f9-48d7-a181-7af3b8ea562c","Type":"ContainerStarted","Data":"7b8d603adae1906b7bf7f3fe6becf97def862deceb62a3933a2e4bd56e7d003d"} Jul 29 21:15:41.953757 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:41.953640 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g" Jul 29 21:15:41.968699 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:41.968658 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q" podStartSLOduration=1.9686457590000002 podStartE2EDuration="1.968645759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:15:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:15:41.966907689 +0000 UTC m=+3037.482213914" watchObservedRunningTime="2026-07-29 21:15:41.968645759 +0000 UTC m=+3037.483951983" Jul 29 21:15:41.981452 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:41.981400 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g" podStartSLOduration=1.9813878740000002 podStartE2EDuration="1.981387874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:15:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:15:41.980643357 +0000 UTC m=+3037.495949620" watchObservedRunningTime="2026-07-29 21:15:41.981387874 +0000 UTC m=+3037.496694099" Jul 29 21:15:51.723101 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:51.723038 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:15:52.961273 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:52.961239 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q" Jul 29 21:15:52.961672 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:15:52.961461 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g" Jul 29 21:16:19.678742 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:19.678712 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5"] Jul 29 21:16:19.681846 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:19.681830 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5" Jul 29 21:16:19.684762 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:19.684737 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-aa96fae3\"" Jul 29 21:16:19.684857 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:19.684826 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-aa96fae3-dockercfg-tkkv6\"" Jul 29 21:16:19.693743 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:19.693719 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8405762b-de5a-46c0-9ecf-e17c7e4bb392-plugins-config-volume\") pod \"payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5\" (UID: \"8405762b-de5a-46c0-9ecf-e17c7e4bb392\") " pod="openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5" Jul 29 21:16:19.693867 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:19.693784 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gbl9\" (UniqueName: \"kubernetes.io/projected/8405762b-de5a-46c0-9ecf-e17c7e4bb392-kube-api-access-6gbl9\") pod \"payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5\" (UID: \"8405762b-de5a-46c0-9ecf-e17c7e4bb392\") " pod="openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5" Jul 29 21:16:19.695439 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:19.695420 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5"] Jul 29 21:16:19.721410 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:19.721389 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h"] Jul 29 21:16:19.724524 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:19.724509 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h" Jul 29 21:16:19.734766 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:19.734740 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h"] Jul 29 21:16:19.794722 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:19.794692 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8405762b-de5a-46c0-9ecf-e17c7e4bb392-plugins-config-volume\") pod \"payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5\" (UID: \"8405762b-de5a-46c0-9ecf-e17c7e4bb392\") " pod="openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5" Jul 29 21:16:19.794888 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:19.794741 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gbl9\" (UniqueName: \"kubernetes.io/projected/8405762b-de5a-46c0-9ecf-e17c7e4bb392-kube-api-access-6gbl9\") pod \"payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5\" (UID: \"8405762b-de5a-46c0-9ecf-e17c7e4bb392\") " pod="openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5" Jul 29 21:16:19.794888 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:19.794785 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7de896c7-e04d-4c9d-a22a-f420701885e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h\" (UID: \"7de896c7-e04d-4c9d-a22a-f420701885e3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h" Jul 29 21:16:19.794888 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:19.794829 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9d8r\" (UniqueName: \"kubernetes.io/projected/7de896c7-e04d-4c9d-a22a-f420701885e3-kube-api-access-c9d8r\") pod \"payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h\" (UID: \"7de896c7-e04d-4c9d-a22a-f420701885e3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h" Jul 29 21:16:19.795437 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:19.795416 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8405762b-de5a-46c0-9ecf-e17c7e4bb392-plugins-config-volume\") pod \"payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5\" (UID: \"8405762b-de5a-46c0-9ecf-e17c7e4bb392\") " pod="openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5" Jul 29 21:16:19.802220 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:19.802197 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gbl9\" (UniqueName: \"kubernetes.io/projected/8405762b-de5a-46c0-9ecf-e17c7e4bb392-kube-api-access-6gbl9\") pod \"payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5\" (UID: \"8405762b-de5a-46c0-9ecf-e17c7e4bb392\") " pod="openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5" Jul 29 21:16:19.896110 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:19.896044 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7de896c7-e04d-4c9d-a22a-f420701885e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h\" (UID: \"7de896c7-e04d-4c9d-a22a-f420701885e3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h" Jul 29 21:16:19.896303 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:19.896140 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9d8r\" (UniqueName: \"kubernetes.io/projected/7de896c7-e04d-4c9d-a22a-f420701885e3-kube-api-access-c9d8r\") pod \"payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h\" (UID: \"7de896c7-e04d-4c9d-a22a-f420701885e3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h" Jul 29 21:16:19.896784 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:19.896763 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7de896c7-e04d-4c9d-a22a-f420701885e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h\" (UID: \"7de896c7-e04d-4c9d-a22a-f420701885e3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h" Jul 29 21:16:19.903610 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:19.903590 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9d8r\" (UniqueName: \"kubernetes.io/projected/7de896c7-e04d-4c9d-a22a-f420701885e3-kube-api-access-c9d8r\") pod \"payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h\" (UID: \"7de896c7-e04d-4c9d-a22a-f420701885e3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h" Jul 29 21:16:19.991430 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:19.991340 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5" Jul 29 21:16:20.034824 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:20.034786 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h" Jul 29 21:16:20.338516 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:20.338486 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5"] Jul 29 21:16:20.341898 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:16:20.341875 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8405762b_de5a_46c0_9ecf_e17c7e4bb392.slice/crio-7ba9bfbfa23775cafa14efbdff04dd5c2967224009982002351738c6cba7608e WatchSource:0}: Error finding container 7ba9bfbfa23775cafa14efbdff04dd5c2967224009982002351738c6cba7608e: Status 404 returned error can't find the container with id 7ba9bfbfa23775cafa14efbdff04dd5c2967224009982002351738c6cba7608e Jul 29 21:16:20.384354 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:20.384326 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h"] Jul 29 21:16:20.386224 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:16:20.386195 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7de896c7_e04d_4c9d_a22a_f420701885e3.slice/crio-b2fa70cb9034b816850fe4faf30474675fadebdb43619604ce34ddb513220b08 WatchSource:0}: Error finding container b2fa70cb9034b816850fe4faf30474675fadebdb43619604ce34ddb513220b08: Status 404 returned error can't find the container with id b2fa70cb9034b816850fe4faf30474675fadebdb43619604ce34ddb513220b08 Jul 29 21:16:21.148343 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:21.148308 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h" event={"ID":"7de896c7-e04d-4c9d-a22a-f420701885e3","Type":"ContainerStarted","Data":"6938ae6ab380031eb4bbd1534b3f0141fadbfb7c4c4fbbe49054a60cc5836d10"} Jul 29 21:16:21.148827 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:21.148472 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h" Jul 29 21:16:21.148827 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:21.148498 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h" event={"ID":"7de896c7-e04d-4c9d-a22a-f420701885e3","Type":"ContainerStarted","Data":"b2fa70cb9034b816850fe4faf30474675fadebdb43619604ce34ddb513220b08"} Jul 29 21:16:21.149813 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:21.149789 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5" event={"ID":"8405762b-de5a-46c0-9ecf-e17c7e4bb392","Type":"ContainerStarted","Data":"029e59f5326ee0563c0cebef236d8dbe1b6339a88e4fd69323e4afee61319210"} Jul 29 21:16:21.149940 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:21.149838 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5" event={"ID":"8405762b-de5a-46c0-9ecf-e17c7e4bb392","Type":"ContainerStarted","Data":"7ba9bfbfa23775cafa14efbdff04dd5c2967224009982002351738c6cba7608e"} Jul 29 21:16:21.149940 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:21.149864 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5" Jul 29 21:16:21.166626 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:21.166585 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h" podStartSLOduration=2.166572945 podStartE2EDuration="2.166572945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:16:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:16:21.164979053 +0000 UTC m=+3076.680285278" watchObservedRunningTime="2026-07-29 21:16:21.166572945 +0000 UTC m=+3076.681879146" Jul 29 21:16:21.179951 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:21.179908 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5" podStartSLOduration=2.179895509 podStartE2EDuration="2.179895509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:16:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:16:21.178406418 +0000 UTC m=+3076.693712643" watchObservedRunningTime="2026-07-29 21:16:21.179895509 +0000 UTC m=+3076.695201733" Jul 29 21:16:26.926924 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:26.926889 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:16:32.155933 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:32.155886 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h" podUID="7de896c7-e04d-4c9d-a22a-f420701885e3" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.48:9004: connect: connection refused" Jul 29 21:16:32.156378 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:32.156324 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5" Jul 29 21:16:38.405042 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:38.405008 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lnvdt"] Jul 29 21:16:38.408526 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:38.408510 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lnvdt" Jul 29 21:16:38.411350 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:38.411325 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-cllsd\"" Jul 29 21:16:38.425178 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:38.425158 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lnvdt"] Jul 29 21:16:38.454378 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:38.454349 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vpxs\" (UniqueName: \"kubernetes.io/projected/7ef57e13-1b09-4ab9-885b-11885fd9adeb-kube-api-access-8vpxs\") pod \"kuadrant-operator-catalog-lnvdt\" (UID: \"7ef57e13-1b09-4ab9-885b-11885fd9adeb\") " pod="kuadrant-system/kuadrant-operator-catalog-lnvdt" Jul 29 21:16:38.555655 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:38.555625 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vpxs\" (UniqueName: \"kubernetes.io/projected/7ef57e13-1b09-4ab9-885b-11885fd9adeb-kube-api-access-8vpxs\") pod \"kuadrant-operator-catalog-lnvdt\" (UID: \"7ef57e13-1b09-4ab9-885b-11885fd9adeb\") " pod="kuadrant-system/kuadrant-operator-catalog-lnvdt" Jul 29 21:16:38.564425 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:38.564404 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vpxs\" (UniqueName: \"kubernetes.io/projected/7ef57e13-1b09-4ab9-885b-11885fd9adeb-kube-api-access-8vpxs\") pod \"kuadrant-operator-catalog-lnvdt\" (UID: \"7ef57e13-1b09-4ab9-885b-11885fd9adeb\") " pod="kuadrant-system/kuadrant-operator-catalog-lnvdt" Jul 29 21:16:38.717836 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:38.717756 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lnvdt" Jul 29 21:16:38.853941 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:38.851117 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lnvdt"] Jul 29 21:16:38.857461 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:16:38.857432 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ef57e13_1b09_4ab9_885b_11885fd9adeb.slice/crio-61acc1cc3eaf04ba21a39f0e0105b22538103e26137436db21ce081ed8bfeac9 WatchSource:0}: Error finding container 61acc1cc3eaf04ba21a39f0e0105b22538103e26137436db21ce081ed8bfeac9: Status 404 returned error can't find the container with id 61acc1cc3eaf04ba21a39f0e0105b22538103e26137436db21ce081ed8bfeac9 Jul 29 21:16:38.859220 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:38.859200 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 21:16:39.242964 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:39.242928 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lnvdt" event={"ID":"7ef57e13-1b09-4ab9-885b-11885fd9adeb","Type":"ContainerStarted","Data":"61acc1cc3eaf04ba21a39f0e0105b22538103e26137436db21ce081ed8bfeac9"} Jul 29 21:16:41.256651 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:41.256588 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lnvdt" event={"ID":"7ef57e13-1b09-4ab9-885b-11885fd9adeb","Type":"ContainerStarted","Data":"4e2432b846447676ec9baf593e13c1dfce6515bf1d44d4a836923d7859f5ef1e"} Jul 29 21:16:41.271817 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:41.271766 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-lnvdt" podStartSLOduration=1.236575919 podStartE2EDuration="3.27174685s" totalImagesPullingTime="2.035170931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:16:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:16:41.270525583 +0000 UTC m=+3096.785831808" watchObservedRunningTime="2026-07-29 21:16:41.27174685 +0000 UTC m=+3096.787053077" Jul 29 21:16:42.156074 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:42.156028 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h" Jul 29 21:16:48.718723 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:48.718684 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-lnvdt" Jul 29 21:16:48.719151 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:48.718935 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-lnvdt" Jul 29 21:16:48.740354 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:48.740323 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-lnvdt" Jul 29 21:16:49.314560 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:49.314533 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-lnvdt" Jul 29 21:16:49.389132 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:49.389092 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lnvdt"] Jul 29 21:16:51.303126 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:51.303086 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-lnvdt" podUID="7ef57e13-1b09-4ab9-885b-11885fd9adeb" containerName="registry-server" containerID="cri-o://4e2432b846447676ec9baf593e13c1dfce6515bf1d44d4a836923d7859f5ef1e" gracePeriod=2 Jul 29 21:16:51.556967 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:51.556902 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lnvdt" Jul 29 21:16:51.672103 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:51.672071 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vpxs\" (UniqueName: \"kubernetes.io/projected/7ef57e13-1b09-4ab9-885b-11885fd9adeb-kube-api-access-8vpxs\") pod \"7ef57e13-1b09-4ab9-885b-11885fd9adeb\" (UID: \"7ef57e13-1b09-4ab9-885b-11885fd9adeb\") " Jul 29 21:16:51.674171 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:51.674148 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ef57e13-1b09-4ab9-885b-11885fd9adeb-kube-api-access-8vpxs" (OuterVolumeSpecName: "kube-api-access-8vpxs") pod "7ef57e13-1b09-4ab9-885b-11885fd9adeb" (UID: "7ef57e13-1b09-4ab9-885b-11885fd9adeb"). InnerVolumeSpecName "kube-api-access-8vpxs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:16:51.772988 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:51.772936 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vpxs\" (UniqueName: \"kubernetes.io/projected/7ef57e13-1b09-4ab9-885b-11885fd9adeb-kube-api-access-8vpxs\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:16:52.309241 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:52.309202 2559 generic.go:356] "Generic (PLEG): container finished" podID="7ef57e13-1b09-4ab9-885b-11885fd9adeb" containerID="4e2432b846447676ec9baf593e13c1dfce6515bf1d44d4a836923d7859f5ef1e" exitCode=0 Jul 29 21:16:52.309658 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:52.309262 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lnvdt" Jul 29 21:16:52.309658 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:52.309290 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lnvdt" event={"ID":"7ef57e13-1b09-4ab9-885b-11885fd9adeb","Type":"ContainerDied","Data":"4e2432b846447676ec9baf593e13c1dfce6515bf1d44d4a836923d7859f5ef1e"} Jul 29 21:16:52.309658 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:52.309346 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lnvdt" event={"ID":"7ef57e13-1b09-4ab9-885b-11885fd9adeb","Type":"ContainerDied","Data":"61acc1cc3eaf04ba21a39f0e0105b22538103e26137436db21ce081ed8bfeac9"} Jul 29 21:16:52.309658 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:52.309367 2559 scope.go:117] "RemoveContainer" containerID="4e2432b846447676ec9baf593e13c1dfce6515bf1d44d4a836923d7859f5ef1e" Jul 29 21:16:52.318507 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:52.318490 2559 scope.go:117] "RemoveContainer" containerID="4e2432b846447676ec9baf593e13c1dfce6515bf1d44d4a836923d7859f5ef1e" Jul 29 21:16:52.318773 ip-10-0-138-67 kubenswrapper[2559]: E0729 21:16:52.318750 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e2432b846447676ec9baf593e13c1dfce6515bf1d44d4a836923d7859f5ef1e\": container with ID starting with 4e2432b846447676ec9baf593e13c1dfce6515bf1d44d4a836923d7859f5ef1e not found: ID does not exist" containerID="4e2432b846447676ec9baf593e13c1dfce6515bf1d44d4a836923d7859f5ef1e" Jul 29 21:16:52.318872 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:52.318777 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e2432b846447676ec9baf593e13c1dfce6515bf1d44d4a836923d7859f5ef1e"} err="failed to get container status \"4e2432b846447676ec9baf593e13c1dfce6515bf1d44d4a836923d7859f5ef1e\": rpc error: code = NotFound desc = could not find container \"4e2432b846447676ec9baf593e13c1dfce6515bf1d44d4a836923d7859f5ef1e\": container with ID starting with 4e2432b846447676ec9baf593e13c1dfce6515bf1d44d4a836923d7859f5ef1e not found: ID does not exist" Jul 29 21:16:52.332110 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:52.332066 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lnvdt"] Jul 29 21:16:52.336466 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:52.336443 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lnvdt"] Jul 29 21:16:53.025533 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:16:53.025499 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ef57e13-1b09-4ab9-885b-11885fd9adeb" path="/var/lib/kubelet/pods/7ef57e13-1b09-4ab9-885b-11885fd9adeb/volumes" Jul 29 21:25:03.431159 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:03.431130 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:25:16.579767 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:16.579719 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5"] Jul 29 21:25:16.580314 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:16.579961 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5" podUID="8405762b-de5a-46c0-9ecf-e17c7e4bb392" containerName="payload-processing" containerID="cri-o://029e59f5326ee0563c0cebef236d8dbe1b6339a88e4fd69323e4afee61319210" gracePeriod=30 Jul 29 21:25:16.586789 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:16.586751 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h"] Jul 29 21:25:16.586986 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:16.586966 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h" podUID="7de896c7-e04d-4c9d-a22a-f420701885e3" containerName="payload-pre-processing" containerID="cri-o://6938ae6ab380031eb4bbd1534b3f0141fadbfb7c4c4fbbe49054a60cc5836d10" gracePeriod=30 Jul 29 21:25:16.847559 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:16.847532 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5" Jul 29 21:25:16.873737 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:16.873711 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h" Jul 29 21:25:16.926749 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:16.926717 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gbl9\" (UniqueName: \"kubernetes.io/projected/8405762b-de5a-46c0-9ecf-e17c7e4bb392-kube-api-access-6gbl9\") pod \"8405762b-de5a-46c0-9ecf-e17c7e4bb392\" (UID: \"8405762b-de5a-46c0-9ecf-e17c7e4bb392\") " Jul 29 21:25:16.926879 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:16.926816 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8405762b-de5a-46c0-9ecf-e17c7e4bb392-plugins-config-volume\") pod \"8405762b-de5a-46c0-9ecf-e17c7e4bb392\" (UID: \"8405762b-de5a-46c0-9ecf-e17c7e4bb392\") " Jul 29 21:25:16.927192 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:16.927171 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8405762b-de5a-46c0-9ecf-e17c7e4bb392-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8405762b-de5a-46c0-9ecf-e17c7e4bb392" (UID: "8405762b-de5a-46c0-9ecf-e17c7e4bb392"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:25:16.928722 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:16.928701 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8405762b-de5a-46c0-9ecf-e17c7e4bb392-kube-api-access-6gbl9" (OuterVolumeSpecName: "kube-api-access-6gbl9") pod "8405762b-de5a-46c0-9ecf-e17c7e4bb392" (UID: "8405762b-de5a-46c0-9ecf-e17c7e4bb392"). InnerVolumeSpecName "kube-api-access-6gbl9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:25:17.027253 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.027221 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9d8r\" (UniqueName: \"kubernetes.io/projected/7de896c7-e04d-4c9d-a22a-f420701885e3-kube-api-access-c9d8r\") pod \"7de896c7-e04d-4c9d-a22a-f420701885e3\" (UID: \"7de896c7-e04d-4c9d-a22a-f420701885e3\") " Jul 29 21:25:17.027421 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.027262 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7de896c7-e04d-4c9d-a22a-f420701885e3-plugins-config-volume\") pod \"7de896c7-e04d-4c9d-a22a-f420701885e3\" (UID: \"7de896c7-e04d-4c9d-a22a-f420701885e3\") " Jul 29 21:25:17.027526 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.027506 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gbl9\" (UniqueName: \"kubernetes.io/projected/8405762b-de5a-46c0-9ecf-e17c7e4bb392-kube-api-access-6gbl9\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:25:17.027587 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.027530 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8405762b-de5a-46c0-9ecf-e17c7e4bb392-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:25:17.027670 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.027653 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7de896c7-e04d-4c9d-a22a-f420701885e3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7de896c7-e04d-4c9d-a22a-f420701885e3" (UID: "7de896c7-e04d-4c9d-a22a-f420701885e3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:25:17.029208 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.029183 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7de896c7-e04d-4c9d-a22a-f420701885e3-kube-api-access-c9d8r" (OuterVolumeSpecName: "kube-api-access-c9d8r") pod "7de896c7-e04d-4c9d-a22a-f420701885e3" (UID: "7de896c7-e04d-4c9d-a22a-f420701885e3"). InnerVolumeSpecName "kube-api-access-c9d8r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:25:17.128844 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.128775 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7de896c7-e04d-4c9d-a22a-f420701885e3-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:25:17.128844 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.128801 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9d8r\" (UniqueName: \"kubernetes.io/projected/7de896c7-e04d-4c9d-a22a-f420701885e3-kube-api-access-c9d8r\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:25:17.669991 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.669953 2559 generic.go:356] "Generic (PLEG): container finished" podID="8405762b-de5a-46c0-9ecf-e17c7e4bb392" containerID="029e59f5326ee0563c0cebef236d8dbe1b6339a88e4fd69323e4afee61319210" exitCode=0 Jul 29 21:25:17.670435 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.670012 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5" Jul 29 21:25:17.670435 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.670071 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5" event={"ID":"8405762b-de5a-46c0-9ecf-e17c7e4bb392","Type":"ContainerDied","Data":"029e59f5326ee0563c0cebef236d8dbe1b6339a88e4fd69323e4afee61319210"} Jul 29 21:25:17.670435 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.670133 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5" event={"ID":"8405762b-de5a-46c0-9ecf-e17c7e4bb392","Type":"ContainerDied","Data":"7ba9bfbfa23775cafa14efbdff04dd5c2967224009982002351738c6cba7608e"} Jul 29 21:25:17.670435 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.670162 2559 scope.go:117] "RemoveContainer" containerID="029e59f5326ee0563c0cebef236d8dbe1b6339a88e4fd69323e4afee61319210" Jul 29 21:25:17.671864 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.671838 2559 generic.go:356] "Generic (PLEG): container finished" podID="7de896c7-e04d-4c9d-a22a-f420701885e3" containerID="6938ae6ab380031eb4bbd1534b3f0141fadbfb7c4c4fbbe49054a60cc5836d10" exitCode=0 Jul 29 21:25:17.671983 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.671907 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h" Jul 29 21:25:17.672277 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.671906 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h" event={"ID":"7de896c7-e04d-4c9d-a22a-f420701885e3","Type":"ContainerDied","Data":"6938ae6ab380031eb4bbd1534b3f0141fadbfb7c4c4fbbe49054a60cc5836d10"} Jul 29 21:25:17.672277 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.672070 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h" event={"ID":"7de896c7-e04d-4c9d-a22a-f420701885e3","Type":"ContainerDied","Data":"b2fa70cb9034b816850fe4faf30474675fadebdb43619604ce34ddb513220b08"} Jul 29 21:25:17.680288 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.680249 2559 scope.go:117] "RemoveContainer" containerID="029e59f5326ee0563c0cebef236d8dbe1b6339a88e4fd69323e4afee61319210" Jul 29 21:25:17.680586 ip-10-0-138-67 kubenswrapper[2559]: E0729 21:25:17.680555 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"029e59f5326ee0563c0cebef236d8dbe1b6339a88e4fd69323e4afee61319210\": container with ID starting with 029e59f5326ee0563c0cebef236d8dbe1b6339a88e4fd69323e4afee61319210 not found: ID does not exist" containerID="029e59f5326ee0563c0cebef236d8dbe1b6339a88e4fd69323e4afee61319210" Jul 29 21:25:17.680666 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.680590 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"029e59f5326ee0563c0cebef236d8dbe1b6339a88e4fd69323e4afee61319210"} err="failed to get container status \"029e59f5326ee0563c0cebef236d8dbe1b6339a88e4fd69323e4afee61319210\": rpc error: code = NotFound desc = could not find container \"029e59f5326ee0563c0cebef236d8dbe1b6339a88e4fd69323e4afee61319210\": container with ID starting with 029e59f5326ee0563c0cebef236d8dbe1b6339a88e4fd69323e4afee61319210 not found: ID does not exist" Jul 29 21:25:17.680666 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.680607 2559 scope.go:117] "RemoveContainer" containerID="6938ae6ab380031eb4bbd1534b3f0141fadbfb7c4c4fbbe49054a60cc5836d10" Jul 29 21:25:17.687969 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.687954 2559 scope.go:117] "RemoveContainer" containerID="6938ae6ab380031eb4bbd1534b3f0141fadbfb7c4c4fbbe49054a60cc5836d10" Jul 29 21:25:17.688272 ip-10-0-138-67 kubenswrapper[2559]: E0729 21:25:17.688243 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6938ae6ab380031eb4bbd1534b3f0141fadbfb7c4c4fbbe49054a60cc5836d10\": container with ID starting with 6938ae6ab380031eb4bbd1534b3f0141fadbfb7c4c4fbbe49054a60cc5836d10 not found: ID does not exist" containerID="6938ae6ab380031eb4bbd1534b3f0141fadbfb7c4c4fbbe49054a60cc5836d10" Jul 29 21:25:17.688368 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.688280 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6938ae6ab380031eb4bbd1534b3f0141fadbfb7c4c4fbbe49054a60cc5836d10"} err="failed to get container status \"6938ae6ab380031eb4bbd1534b3f0141fadbfb7c4c4fbbe49054a60cc5836d10\": rpc error: code = NotFound desc = could not find container \"6938ae6ab380031eb4bbd1534b3f0141fadbfb7c4c4fbbe49054a60cc5836d10\": container with ID starting with 6938ae6ab380031eb4bbd1534b3f0141fadbfb7c4c4fbbe49054a60cc5836d10 not found: ID does not exist" Jul 29 21:25:17.690358 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.690339 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5"] Jul 29 21:25:17.694553 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.694534 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-aa96fae3-84d6d4b798-98mg5"] Jul 29 21:25:17.709241 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.709223 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h"] Jul 29 21:25:17.712988 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:17.712969 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-aa96fae3-99f58bfc6-dk57h"] Jul 29 21:25:19.025694 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:19.025657 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7de896c7-e04d-4c9d-a22a-f420701885e3" path="/var/lib/kubelet/pods/7de896c7-e04d-4c9d-a22a-f420701885e3/volumes" Jul 29 21:25:19.026103 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:19.026005 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8405762b-de5a-46c0-9ecf-e17c7e4bb392" path="/var/lib/kubelet/pods/8405762b-de5a-46c0-9ecf-e17c7e4bb392/volumes" Jul 29 21:25:19.133312 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:25:19.133281 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:26:01.795554 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:01.795520 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q"] Jul 29 21:26:01.796190 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:01.795728 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q" podUID="64a7386b-6e4e-455b-a662-f80f2341c7d6" containerName="payload-processing" containerID="cri-o://e59dab8de08810759da87fa98fa373204914b033aba282fcaae69a770a60cd7a" gracePeriod=30 Jul 29 21:26:01.804754 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:01.804726 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g"] Jul 29 21:26:01.805147 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:01.805114 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g" podUID="a2c62560-c0f9-48d7-a181-7af3b8ea562c" containerName="payload-pre-processing" containerID="cri-o://7b8d603adae1906b7bf7f3fe6becf97def862deceb62a3933a2e4bd56e7d003d" gracePeriod=30 Jul 29 21:26:02.078917 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.078895 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q" Jul 29 21:26:02.081694 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.081676 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g" Jul 29 21:26:02.220259 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.220225 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2c62560-c0f9-48d7-a181-7af3b8ea562c-plugins-config-volume\") pod \"a2c62560-c0f9-48d7-a181-7af3b8ea562c\" (UID: \"a2c62560-c0f9-48d7-a181-7af3b8ea562c\") " Jul 29 21:26:02.220259 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.220264 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64a7386b-6e4e-455b-a662-f80f2341c7d6-plugins-config-volume\") pod \"64a7386b-6e4e-455b-a662-f80f2341c7d6\" (UID: \"64a7386b-6e4e-455b-a662-f80f2341c7d6\") " Jul 29 21:26:02.220522 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.220315 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bd7jr\" (UniqueName: \"kubernetes.io/projected/64a7386b-6e4e-455b-a662-f80f2341c7d6-kube-api-access-bd7jr\") pod \"64a7386b-6e4e-455b-a662-f80f2341c7d6\" (UID: \"64a7386b-6e4e-455b-a662-f80f2341c7d6\") " Jul 29 21:26:02.220522 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.220363 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8vzj\" (UniqueName: \"kubernetes.io/projected/a2c62560-c0f9-48d7-a181-7af3b8ea562c-kube-api-access-t8vzj\") pod \"a2c62560-c0f9-48d7-a181-7af3b8ea562c\" (UID: \"a2c62560-c0f9-48d7-a181-7af3b8ea562c\") " Jul 29 21:26:02.220714 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.220690 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2c62560-c0f9-48d7-a181-7af3b8ea562c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a2c62560-c0f9-48d7-a181-7af3b8ea562c" (UID: "a2c62560-c0f9-48d7-a181-7af3b8ea562c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:26:02.220778 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.220758 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64a7386b-6e4e-455b-a662-f80f2341c7d6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "64a7386b-6e4e-455b-a662-f80f2341c7d6" (UID: "64a7386b-6e4e-455b-a662-f80f2341c7d6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:26:02.222455 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.222431 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2c62560-c0f9-48d7-a181-7af3b8ea562c-kube-api-access-t8vzj" (OuterVolumeSpecName: "kube-api-access-t8vzj") pod "a2c62560-c0f9-48d7-a181-7af3b8ea562c" (UID: "a2c62560-c0f9-48d7-a181-7af3b8ea562c"). InnerVolumeSpecName "kube-api-access-t8vzj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:26:02.222562 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.222486 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64a7386b-6e4e-455b-a662-f80f2341c7d6-kube-api-access-bd7jr" (OuterVolumeSpecName: "kube-api-access-bd7jr") pod "64a7386b-6e4e-455b-a662-f80f2341c7d6" (UID: "64a7386b-6e4e-455b-a662-f80f2341c7d6"). InnerVolumeSpecName "kube-api-access-bd7jr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:26:02.321454 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.321371 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2c62560-c0f9-48d7-a181-7af3b8ea562c-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:26:02.321454 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.321401 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64a7386b-6e4e-455b-a662-f80f2341c7d6-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:26:02.321454 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.321410 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bd7jr\" (UniqueName: \"kubernetes.io/projected/64a7386b-6e4e-455b-a662-f80f2341c7d6-kube-api-access-bd7jr\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:26:02.321454 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.321419 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8vzj\" (UniqueName: \"kubernetes.io/projected/a2c62560-c0f9-48d7-a181-7af3b8ea562c-kube-api-access-t8vzj\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:26:02.891898 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.891863 2559 generic.go:356] "Generic (PLEG): container finished" podID="64a7386b-6e4e-455b-a662-f80f2341c7d6" containerID="e59dab8de08810759da87fa98fa373204914b033aba282fcaae69a770a60cd7a" exitCode=0 Jul 29 21:26:02.892400 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.891927 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q" Jul 29 21:26:02.892400 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.891934 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q" event={"ID":"64a7386b-6e4e-455b-a662-f80f2341c7d6","Type":"ContainerDied","Data":"e59dab8de08810759da87fa98fa373204914b033aba282fcaae69a770a60cd7a"} Jul 29 21:26:02.892400 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.892002 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q" event={"ID":"64a7386b-6e4e-455b-a662-f80f2341c7d6","Type":"ContainerDied","Data":"4168dcdde1e46a99e07042dd7899ac21a23c6951f8135b0ba080a78db427a03d"} Jul 29 21:26:02.892400 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.892030 2559 scope.go:117] "RemoveContainer" containerID="e59dab8de08810759da87fa98fa373204914b033aba282fcaae69a770a60cd7a" Jul 29 21:26:02.893425 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.893347 2559 generic.go:356] "Generic (PLEG): container finished" podID="a2c62560-c0f9-48d7-a181-7af3b8ea562c" containerID="7b8d603adae1906b7bf7f3fe6becf97def862deceb62a3933a2e4bd56e7d003d" exitCode=0 Jul 29 21:26:02.893425 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.893392 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g" Jul 29 21:26:02.893425 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.893419 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g" event={"ID":"a2c62560-c0f9-48d7-a181-7af3b8ea562c","Type":"ContainerDied","Data":"7b8d603adae1906b7bf7f3fe6becf97def862deceb62a3933a2e4bd56e7d003d"} Jul 29 21:26:02.893596 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.893452 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g" event={"ID":"a2c62560-c0f9-48d7-a181-7af3b8ea562c","Type":"ContainerDied","Data":"0a532bb9568921712491d36acd2f7bf00744a12084f0c94086c77ec6d056585a"} Jul 29 21:26:02.901189 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.901104 2559 scope.go:117] "RemoveContainer" containerID="e59dab8de08810759da87fa98fa373204914b033aba282fcaae69a770a60cd7a" Jul 29 21:26:02.901427 ip-10-0-138-67 kubenswrapper[2559]: E0729 21:26:02.901407 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e59dab8de08810759da87fa98fa373204914b033aba282fcaae69a770a60cd7a\": container with ID starting with e59dab8de08810759da87fa98fa373204914b033aba282fcaae69a770a60cd7a not found: ID does not exist" containerID="e59dab8de08810759da87fa98fa373204914b033aba282fcaae69a770a60cd7a" Jul 29 21:26:02.901493 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.901436 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e59dab8de08810759da87fa98fa373204914b033aba282fcaae69a770a60cd7a"} err="failed to get container status \"e59dab8de08810759da87fa98fa373204914b033aba282fcaae69a770a60cd7a\": rpc error: code = NotFound desc = could not find container \"e59dab8de08810759da87fa98fa373204914b033aba282fcaae69a770a60cd7a\": container with ID starting with e59dab8de08810759da87fa98fa373204914b033aba282fcaae69a770a60cd7a not found: ID does not exist" Jul 29 21:26:02.901493 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.901452 2559 scope.go:117] "RemoveContainer" containerID="7b8d603adae1906b7bf7f3fe6becf97def862deceb62a3933a2e4bd56e7d003d" Jul 29 21:26:02.908956 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.908940 2559 scope.go:117] "RemoveContainer" containerID="7b8d603adae1906b7bf7f3fe6becf97def862deceb62a3933a2e4bd56e7d003d" Jul 29 21:26:02.909210 ip-10-0-138-67 kubenswrapper[2559]: E0729 21:26:02.909192 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b8d603adae1906b7bf7f3fe6becf97def862deceb62a3933a2e4bd56e7d003d\": container with ID starting with 7b8d603adae1906b7bf7f3fe6becf97def862deceb62a3933a2e4bd56e7d003d not found: ID does not exist" containerID="7b8d603adae1906b7bf7f3fe6becf97def862deceb62a3933a2e4bd56e7d003d" Jul 29 21:26:02.909259 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.909218 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b8d603adae1906b7bf7f3fe6becf97def862deceb62a3933a2e4bd56e7d003d"} err="failed to get container status \"7b8d603adae1906b7bf7f3fe6becf97def862deceb62a3933a2e4bd56e7d003d\": rpc error: code = NotFound desc = could not find container \"7b8d603adae1906b7bf7f3fe6becf97def862deceb62a3933a2e4bd56e7d003d\": container with ID starting with 7b8d603adae1906b7bf7f3fe6becf97def862deceb62a3933a2e4bd56e7d003d not found: ID does not exist" Jul 29 21:26:02.935554 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.935532 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g"] Jul 29 21:26:02.944687 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.944668 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fcb1aff7-cf49447f6-gbl8g"] Jul 29 21:26:02.966308 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.966271 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q"] Jul 29 21:26:02.970117 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:02.970096 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fcb1aff7-77f95cd47d-xzx2q"] Jul 29 21:26:03.027403 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:03.027373 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64a7386b-6e4e-455b-a662-f80f2341c7d6" path="/var/lib/kubelet/pods/64a7386b-6e4e-455b-a662-f80f2341c7d6/volumes" Jul 29 21:26:03.027724 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:03.027711 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2c62560-c0f9-48d7-a181-7af3b8ea562c" path="/var/lib/kubelet/pods/a2c62560-c0f9-48d7-a181-7af3b8ea562c/volumes" Jul 29 21:26:06.801237 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.801199 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz"] Jul 29 21:26:06.801615 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.801528 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2c62560-c0f9-48d7-a181-7af3b8ea562c" containerName="payload-pre-processing" Jul 29 21:26:06.801615 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.801539 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2c62560-c0f9-48d7-a181-7af3b8ea562c" containerName="payload-pre-processing" Jul 29 21:26:06.801615 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.801553 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64a7386b-6e4e-455b-a662-f80f2341c7d6" containerName="payload-processing" Jul 29 21:26:06.801615 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.801558 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="64a7386b-6e4e-455b-a662-f80f2341c7d6" containerName="payload-processing" Jul 29 21:26:06.801615 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.801566 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7de896c7-e04d-4c9d-a22a-f420701885e3" containerName="payload-pre-processing" Jul 29 21:26:06.801615 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.801571 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7de896c7-e04d-4c9d-a22a-f420701885e3" containerName="payload-pre-processing" Jul 29 21:26:06.801615 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.801579 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8405762b-de5a-46c0-9ecf-e17c7e4bb392" containerName="payload-processing" Jul 29 21:26:06.801615 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.801585 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8405762b-de5a-46c0-9ecf-e17c7e4bb392" containerName="payload-processing" Jul 29 21:26:06.801615 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.801594 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ef57e13-1b09-4ab9-885b-11885fd9adeb" containerName="registry-server" Jul 29 21:26:06.801615 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.801600 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ef57e13-1b09-4ab9-885b-11885fd9adeb" containerName="registry-server" Jul 29 21:26:06.801931 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.801649 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="64a7386b-6e4e-455b-a662-f80f2341c7d6" containerName="payload-processing" Jul 29 21:26:06.801931 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.801659 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ef57e13-1b09-4ab9-885b-11885fd9adeb" containerName="registry-server" Jul 29 21:26:06.801931 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.801665 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8405762b-de5a-46c0-9ecf-e17c7e4bb392" containerName="payload-processing" Jul 29 21:26:06.801931 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.801673 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2c62560-c0f9-48d7-a181-7af3b8ea562c" containerName="payload-pre-processing" Jul 29 21:26:06.801931 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.801683 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7de896c7-e04d-4c9d-a22a-f420701885e3" containerName="payload-pre-processing" Jul 29 21:26:06.806386 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.806365 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz" Jul 29 21:26:06.808856 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.808837 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-b04ec1\"" Jul 29 21:26:06.809099 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.809082 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-b04ec1-dockercfg-9ppzn\"" Jul 29 21:26:06.817294 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.817262 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz"] Jul 29 21:26:06.968389 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.963814 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxwtp\" (UniqueName: \"kubernetes.io/projected/4b38e3b4-056f-4515-ae79-705ee6e0508a-kube-api-access-gxwtp\") pod \"payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz\" (UID: \"4b38e3b4-056f-4515-ae79-705ee6e0508a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz" Jul 29 21:26:06.968389 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:06.963908 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b38e3b4-056f-4515-ae79-705ee6e0508a-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz\" (UID: \"4b38e3b4-056f-4515-ae79-705ee6e0508a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz" Jul 29 21:26:07.064515 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:07.064432 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxwtp\" (UniqueName: \"kubernetes.io/projected/4b38e3b4-056f-4515-ae79-705ee6e0508a-kube-api-access-gxwtp\") pod \"payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz\" (UID: \"4b38e3b4-056f-4515-ae79-705ee6e0508a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz" Jul 29 21:26:07.064515 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:07.064478 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b38e3b4-056f-4515-ae79-705ee6e0508a-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz\" (UID: \"4b38e3b4-056f-4515-ae79-705ee6e0508a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz" Jul 29 21:26:07.065239 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:07.065214 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b38e3b4-056f-4515-ae79-705ee6e0508a-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz\" (UID: \"4b38e3b4-056f-4515-ae79-705ee6e0508a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz" Jul 29 21:26:07.078956 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:07.078933 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxwtp\" (UniqueName: \"kubernetes.io/projected/4b38e3b4-056f-4515-ae79-705ee6e0508a-kube-api-access-gxwtp\") pod \"payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz\" (UID: \"4b38e3b4-056f-4515-ae79-705ee6e0508a\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz" Jul 29 21:26:07.117015 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:07.116990 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz" Jul 29 21:26:07.250291 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:07.250259 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz"] Jul 29 21:26:07.254152 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:26:07.254121 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b38e3b4_056f_4515_ae79_705ee6e0508a.slice/crio-5d3a99dce51797fe517e1cf71b2c3bf4dec2a197c0aaba18466f970956ebc919 WatchSource:0}: Error finding container 5d3a99dce51797fe517e1cf71b2c3bf4dec2a197c0aaba18466f970956ebc919: Status 404 returned error can't find the container with id 5d3a99dce51797fe517e1cf71b2c3bf4dec2a197c0aaba18466f970956ebc919 Jul 29 21:26:07.920353 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:07.920318 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz" event={"ID":"4b38e3b4-056f-4515-ae79-705ee6e0508a","Type":"ContainerStarted","Data":"b1308508e4e2cd4a092210ef0ec31a076c8bb16c6ebf5bb6cc3c4ba0f1163511"} Jul 29 21:26:07.920855 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:07.920376 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz" event={"ID":"4b38e3b4-056f-4515-ae79-705ee6e0508a","Type":"ContainerStarted","Data":"5d3a99dce51797fe517e1cf71b2c3bf4dec2a197c0aaba18466f970956ebc919"} Jul 29 21:26:07.920855 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:07.920401 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz" Jul 29 21:26:07.937490 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:07.937441 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz" podStartSLOduration=1.937421313 podStartE2EDuration="1.937421313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:26:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:26:07.93631794 +0000 UTC m=+3663.451624171" watchObservedRunningTime="2026-07-29 21:26:07.937421313 +0000 UTC m=+3663.452727538" Jul 29 21:26:10.137250 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:10.137218 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:26:18.930158 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:18.930125 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz" Jul 29 21:26:35.980201 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:35.980125 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq"] Jul 29 21:26:35.983447 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:35.983432 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq" Jul 29 21:26:35.986432 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:35.986415 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-46f813\"" Jul 29 21:26:35.996929 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:35.996905 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq"] Jul 29 21:26:36.003557 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:36.003531 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7w4z\" (UniqueName: \"kubernetes.io/projected/cf5d4e51-850c-428f-ac8b-62dbfb1516fb-kube-api-access-p7w4z\") pod \"payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq\" (UID: \"cf5d4e51-850c-428f-ac8b-62dbfb1516fb\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq" Jul 29 21:26:36.003669 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:36.003653 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf5d4e51-850c-428f-ac8b-62dbfb1516fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq\" (UID: \"cf5d4e51-850c-428f-ac8b-62dbfb1516fb\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq" Jul 29 21:26:36.014409 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:36.014386 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz"] Jul 29 21:26:36.017687 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:36.017673 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz" Jul 29 21:26:36.020183 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:36.020164 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-46f813-dockercfg-5ngzr\"" Jul 29 21:26:36.028767 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:36.028747 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz"] Jul 29 21:26:36.104136 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:36.104103 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97a43b3e-d3c3-41b7-b1d1-349f5d1d7042-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz\" (UID: \"97a43b3e-d3c3-41b7-b1d1-349f5d1d7042\") " pod="openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz" Jul 29 21:26:36.104304 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:36.104156 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blgsn\" (UniqueName: \"kubernetes.io/projected/97a43b3e-d3c3-41b7-b1d1-349f5d1d7042-kube-api-access-blgsn\") pod \"payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz\" (UID: \"97a43b3e-d3c3-41b7-b1d1-349f5d1d7042\") " pod="openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz" Jul 29 21:26:36.104357 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:36.104300 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf5d4e51-850c-428f-ac8b-62dbfb1516fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq\" (UID: \"cf5d4e51-850c-428f-ac8b-62dbfb1516fb\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq" Jul 29 21:26:36.104483 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:36.104460 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7w4z\" (UniqueName: \"kubernetes.io/projected/cf5d4e51-850c-428f-ac8b-62dbfb1516fb-kube-api-access-p7w4z\") pod \"payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq\" (UID: \"cf5d4e51-850c-428f-ac8b-62dbfb1516fb\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq" Jul 29 21:26:36.105123 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:36.105102 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf5d4e51-850c-428f-ac8b-62dbfb1516fb-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq\" (UID: \"cf5d4e51-850c-428f-ac8b-62dbfb1516fb\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq" Jul 29 21:26:36.113802 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:36.113740 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7w4z\" (UniqueName: \"kubernetes.io/projected/cf5d4e51-850c-428f-ac8b-62dbfb1516fb-kube-api-access-p7w4z\") pod \"payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq\" (UID: \"cf5d4e51-850c-428f-ac8b-62dbfb1516fb\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq" Jul 29 21:26:36.204968 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:36.204943 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97a43b3e-d3c3-41b7-b1d1-349f5d1d7042-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz\" (UID: \"97a43b3e-d3c3-41b7-b1d1-349f5d1d7042\") " pod="openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz" Jul 29 21:26:36.205133 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:36.204975 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blgsn\" (UniqueName: \"kubernetes.io/projected/97a43b3e-d3c3-41b7-b1d1-349f5d1d7042-kube-api-access-blgsn\") pod \"payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz\" (UID: \"97a43b3e-d3c3-41b7-b1d1-349f5d1d7042\") " pod="openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz" Jul 29 21:26:36.205603 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:36.205584 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97a43b3e-d3c3-41b7-b1d1-349f5d1d7042-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz\" (UID: \"97a43b3e-d3c3-41b7-b1d1-349f5d1d7042\") " pod="openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz" Jul 29 21:26:36.213941 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:36.213916 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blgsn\" (UniqueName: \"kubernetes.io/projected/97a43b3e-d3c3-41b7-b1d1-349f5d1d7042-kube-api-access-blgsn\") pod \"payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz\" (UID: \"97a43b3e-d3c3-41b7-b1d1-349f5d1d7042\") " pod="openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz" Jul 29 21:26:36.293655 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:36.293622 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq" Jul 29 21:26:36.327144 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:36.327108 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz" Jul 29 21:26:36.468125 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:36.468101 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz"] Jul 29 21:26:36.469966 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:26:36.469936 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97a43b3e_d3c3_41b7_b1d1_349f5d1d7042.slice/crio-4b87a941b0df1b82504ff60df4f8a5240e7fb7a47b91d5fc2e6c735a9eb9e53a WatchSource:0}: Error finding container 4b87a941b0df1b82504ff60df4f8a5240e7fb7a47b91d5fc2e6c735a9eb9e53a: Status 404 returned error can't find the container with id 4b87a941b0df1b82504ff60df4f8a5240e7fb7a47b91d5fc2e6c735a9eb9e53a Jul 29 21:26:36.647220 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:36.647193 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq"] Jul 29 21:26:36.650017 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:26:36.649975 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf5d4e51_850c_428f_ac8b_62dbfb1516fb.slice/crio-051a658458e3b51f1c5c903b1404797cea1ef989331abd6e0f36ebc63e0adcf0 WatchSource:0}: Error finding container 051a658458e3b51f1c5c903b1404797cea1ef989331abd6e0f36ebc63e0adcf0: Status 404 returned error can't find the container with id 051a658458e3b51f1c5c903b1404797cea1ef989331abd6e0f36ebc63e0adcf0 Jul 29 21:26:37.057019 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:37.056979 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq" event={"ID":"cf5d4e51-850c-428f-ac8b-62dbfb1516fb","Type":"ContainerStarted","Data":"04902041b93edb199fc1c55d73c3bccce0775da4bde743e2e90815e2668dff2d"} Jul 29 21:26:37.057498 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:37.057039 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq" Jul 29 21:26:37.057498 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:37.057088 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq" event={"ID":"cf5d4e51-850c-428f-ac8b-62dbfb1516fb","Type":"ContainerStarted","Data":"051a658458e3b51f1c5c903b1404797cea1ef989331abd6e0f36ebc63e0adcf0"} Jul 29 21:26:37.058490 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:37.058467 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz" event={"ID":"97a43b3e-d3c3-41b7-b1d1-349f5d1d7042","Type":"ContainerStarted","Data":"168902d95cf2960fb0830f06a446b95c637eaa515565be5068d5480658790cee"} Jul 29 21:26:37.058599 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:37.058512 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz" Jul 29 21:26:37.058599 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:37.058530 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz" event={"ID":"97a43b3e-d3c3-41b7-b1d1-349f5d1d7042","Type":"ContainerStarted","Data":"4b87a941b0df1b82504ff60df4f8a5240e7fb7a47b91d5fc2e6c735a9eb9e53a"} Jul 29 21:26:37.074794 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:37.074748 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq" podStartSLOduration=2.074734661 podStartE2EDuration="2.074734661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:26:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:26:37.074169155 +0000 UTC m=+3692.589475378" watchObservedRunningTime="2026-07-29 21:26:37.074734661 +0000 UTC m=+3692.590040885" Jul 29 21:26:37.093380 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:37.093329 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz" podStartSLOduration=2.093310256 podStartE2EDuration="2.093310256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:26:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:26:37.092573046 +0000 UTC m=+3692.607879294" watchObservedRunningTime="2026-07-29 21:26:37.093310256 +0000 UTC m=+3692.608616458" Jul 29 21:26:41.930924 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:41.930890 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:26:48.070600 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:48.070569 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz" Jul 29 21:26:48.071087 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:26:48.070922 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq" Jul 29 21:27:09.745614 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:27:09.745580 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:27:52.669607 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:27:52.669578 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:28:36.407577 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:36.407542 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b04ec1-pc5cl"] Jul 29 21:28:36.410928 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:36.410911 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b04ec1-pc5cl" Jul 29 21:28:36.416761 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:36.416742 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 29 21:28:36.417272 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:36.416967 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wpqrc\"" Jul 29 21:28:36.424113 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:36.424090 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b04ec1-pc5cl"] Jul 29 21:28:36.533454 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:36.533427 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8e97229-9126-442f-a329-2682c40d9d58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-b04ec1-pc5cl\" (UID: \"c8e97229-9126-442f-a329-2682c40d9d58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b04ec1-pc5cl" Jul 29 21:28:36.634281 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:36.634246 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8e97229-9126-442f-a329-2682c40d9d58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-b04ec1-pc5cl\" (UID: \"c8e97229-9126-442f-a329-2682c40d9d58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b04ec1-pc5cl" Jul 29 21:28:36.634836 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:36.634813 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8e97229-9126-442f-a329-2682c40d9d58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-b04ec1-pc5cl\" (UID: \"c8e97229-9126-442f-a329-2682c40d9d58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b04ec1-pc5cl" Jul 29 21:28:36.720401 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:36.720335 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b04ec1-pc5cl" Jul 29 21:28:37.073853 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:37.073830 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b04ec1-pc5cl"] Jul 29 21:28:37.076301 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:28:37.076273 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8e97229_9126_442f_a329_2682c40d9d58.slice/crio-585f11dcb44434a075e78cd7a742d3c92607a244299fa8459b7ae08d50174e1c WatchSource:0}: Error finding container 585f11dcb44434a075e78cd7a742d3c92607a244299fa8459b7ae08d50174e1c: Status 404 returned error can't find the container with id 585f11dcb44434a075e78cd7a742d3c92607a244299fa8459b7ae08d50174e1c Jul 29 21:28:37.627279 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:37.627245 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b04ec1-pc5cl" event={"ID":"c8e97229-9126-442f-a329-2682c40d9d58","Type":"ContainerStarted","Data":"e472a093ee50205fa9a7d87cd36c8768bf5fedc42e26223b919cfdf325eaff59"} Jul 29 21:28:37.627664 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:37.627305 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b04ec1-pc5cl" event={"ID":"c8e97229-9126-442f-a329-2682c40d9d58","Type":"ContainerStarted","Data":"585f11dcb44434a075e78cd7a742d3c92607a244299fa8459b7ae08d50174e1c"} Jul 29 21:28:37.652321 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:37.652275 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b04ec1-pc5cl" podStartSLOduration=1.652260061 podStartE2EDuration="1.652260061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:28:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:28:37.649944603 +0000 UTC m=+3813.165250827" watchObservedRunningTime="2026-07-29 21:28:37.652260061 +0000 UTC m=+3813.167566285" Jul 29 21:28:38.633227 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:38.633193 2559 generic.go:356] "Generic (PLEG): container finished" podID="c8e97229-9126-442f-a329-2682c40d9d58" containerID="e472a093ee50205fa9a7d87cd36c8768bf5fedc42e26223b919cfdf325eaff59" exitCode=0 Jul 29 21:28:38.633636 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:38.633278 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b04ec1-pc5cl" event={"ID":"c8e97229-9126-442f-a329-2682c40d9d58","Type":"ContainerDied","Data":"e472a093ee50205fa9a7d87cd36c8768bf5fedc42e26223b919cfdf325eaff59"} Jul 29 21:28:39.752982 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:39.752957 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:28:39.769462 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:39.769438 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b04ec1-pc5cl" Jul 29 21:28:39.862957 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:39.862927 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8e97229-9126-442f-a329-2682c40d9d58-maas-api-service-ca\") pod \"c8e97229-9126-442f-a329-2682c40d9d58\" (UID: \"c8e97229-9126-442f-a329-2682c40d9d58\") " Jul 29 21:28:39.863300 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:39.863279 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8e97229-9126-442f-a329-2682c40d9d58-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c8e97229-9126-442f-a329-2682c40d9d58" (UID: "c8e97229-9126-442f-a329-2682c40d9d58"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:28:39.963998 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:39.963917 2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8e97229-9126-442f-a329-2682c40d9d58-maas-api-service-ca\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:28:40.642978 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:40.642952 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b04ec1-pc5cl" Jul 29 21:28:40.642978 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:40.642967 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b04ec1-pc5cl" event={"ID":"c8e97229-9126-442f-a329-2682c40d9d58","Type":"ContainerDied","Data":"585f11dcb44434a075e78cd7a742d3c92607a244299fa8459b7ae08d50174e1c"} Jul 29 21:28:40.643215 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:40.643009 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="585f11dcb44434a075e78cd7a742d3c92607a244299fa8459b7ae08d50174e1c" Jul 29 21:28:47.155701 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:47.155667 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz"] Jul 29 21:28:47.156130 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:47.155837 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz" podUID="4b38e3b4-056f-4515-ae79-705ee6e0508a" containerName="payload-processing" containerID="cri-o://b1308508e4e2cd4a092210ef0ec31a076c8bb16c6ebf5bb6cc3c4ba0f1163511" gracePeriod=30 Jul 29 21:28:47.409921 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:47.409859 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz" Jul 29 21:28:47.528691 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:47.528660 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxwtp\" (UniqueName: \"kubernetes.io/projected/4b38e3b4-056f-4515-ae79-705ee6e0508a-kube-api-access-gxwtp\") pod \"4b38e3b4-056f-4515-ae79-705ee6e0508a\" (UID: \"4b38e3b4-056f-4515-ae79-705ee6e0508a\") " Jul 29 21:28:47.528856 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:47.528698 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b38e3b4-056f-4515-ae79-705ee6e0508a-plugins-config-volume\") pod \"4b38e3b4-056f-4515-ae79-705ee6e0508a\" (UID: \"4b38e3b4-056f-4515-ae79-705ee6e0508a\") " Jul 29 21:28:47.529098 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:47.529066 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b38e3b4-056f-4515-ae79-705ee6e0508a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4b38e3b4-056f-4515-ae79-705ee6e0508a" (UID: "4b38e3b4-056f-4515-ae79-705ee6e0508a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:28:47.530893 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:47.530867 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b38e3b4-056f-4515-ae79-705ee6e0508a-kube-api-access-gxwtp" (OuterVolumeSpecName: "kube-api-access-gxwtp") pod "4b38e3b4-056f-4515-ae79-705ee6e0508a" (UID: "4b38e3b4-056f-4515-ae79-705ee6e0508a"). InnerVolumeSpecName "kube-api-access-gxwtp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:28:47.598979 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:47.598956 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b04ec1-pc5cl"] Jul 29 21:28:47.606099 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:47.606074 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b04ec1-pc5cl"] Jul 29 21:28:47.629622 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:47.629601 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxwtp\" (UniqueName: \"kubernetes.io/projected/4b38e3b4-056f-4515-ae79-705ee6e0508a-kube-api-access-gxwtp\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:28:47.629622 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:47.629622 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b38e3b4-056f-4515-ae79-705ee6e0508a-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:28:47.677888 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:47.677820 2559 generic.go:356] "Generic (PLEG): container finished" podID="4b38e3b4-056f-4515-ae79-705ee6e0508a" containerID="b1308508e4e2cd4a092210ef0ec31a076c8bb16c6ebf5bb6cc3c4ba0f1163511" exitCode=0 Jul 29 21:28:47.677888 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:47.677867 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz" Jul 29 21:28:47.678073 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:47.677906 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz" event={"ID":"4b38e3b4-056f-4515-ae79-705ee6e0508a","Type":"ContainerDied","Data":"b1308508e4e2cd4a092210ef0ec31a076c8bb16c6ebf5bb6cc3c4ba0f1163511"} Jul 29 21:28:47.678073 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:47.677996 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz" event={"ID":"4b38e3b4-056f-4515-ae79-705ee6e0508a","Type":"ContainerDied","Data":"5d3a99dce51797fe517e1cf71b2c3bf4dec2a197c0aaba18466f970956ebc919"} Jul 29 21:28:47.678073 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:47.678033 2559 scope.go:117] "RemoveContainer" containerID="b1308508e4e2cd4a092210ef0ec31a076c8bb16c6ebf5bb6cc3c4ba0f1163511" Jul 29 21:28:47.687116 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:47.687101 2559 scope.go:117] "RemoveContainer" containerID="b1308508e4e2cd4a092210ef0ec31a076c8bb16c6ebf5bb6cc3c4ba0f1163511" Jul 29 21:28:47.687359 ip-10-0-138-67 kubenswrapper[2559]: E0729 21:28:47.687340 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1308508e4e2cd4a092210ef0ec31a076c8bb16c6ebf5bb6cc3c4ba0f1163511\": container with ID starting with b1308508e4e2cd4a092210ef0ec31a076c8bb16c6ebf5bb6cc3c4ba0f1163511 not found: ID does not exist" containerID="b1308508e4e2cd4a092210ef0ec31a076c8bb16c6ebf5bb6cc3c4ba0f1163511" Jul 29 21:28:47.687425 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:47.687365 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1308508e4e2cd4a092210ef0ec31a076c8bb16c6ebf5bb6cc3c4ba0f1163511"} err="failed to get container status \"b1308508e4e2cd4a092210ef0ec31a076c8bb16c6ebf5bb6cc3c4ba0f1163511\": rpc error: code = NotFound desc = could not find container \"b1308508e4e2cd4a092210ef0ec31a076c8bb16c6ebf5bb6cc3c4ba0f1163511\": container with ID starting with b1308508e4e2cd4a092210ef0ec31a076c8bb16c6ebf5bb6cc3c4ba0f1163511 not found: ID does not exist" Jul 29 21:28:47.710682 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:47.710663 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz"] Jul 29 21:28:47.718249 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:47.718231 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b04ec1-5c5887bbfb-x78wz"] Jul 29 21:28:49.025365 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:49.025333 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b38e3b4-056f-4515-ae79-705ee6e0508a" path="/var/lib/kubelet/pods/4b38e3b4-056f-4515-ae79-705ee6e0508a/volumes" Jul 29 21:28:49.025778 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:49.025637 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8e97229-9126-442f-a329-2682c40d9d58" path="/var/lib/kubelet/pods/c8e97229-9126-442f-a329-2682c40d9d58/volumes" Jul 29 21:28:51.739813 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:28:51.739781 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:29:05.583269 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.583184 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz"] Jul 29 21:29:05.583726 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.583377 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz" podUID="97a43b3e-d3c3-41b7-b1d1-349f5d1d7042" containerName="payload-processing" containerID="cri-o://168902d95cf2960fb0830f06a446b95c637eaa515565be5068d5480658790cee" gracePeriod=30 Jul 29 21:29:05.588377 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.588350 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq"] Jul 29 21:29:05.588783 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.588730 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq" podUID="cf5d4e51-850c-428f-ac8b-62dbfb1516fb" containerName="payload-pre-processing" containerID="cri-o://04902041b93edb199fc1c55d73c3bccce0775da4bde743e2e90815e2668dff2d" gracePeriod=30 Jul 29 21:29:05.770012 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.769929 2559 generic.go:356] "Generic (PLEG): container finished" podID="cf5d4e51-850c-428f-ac8b-62dbfb1516fb" containerID="04902041b93edb199fc1c55d73c3bccce0775da4bde743e2e90815e2668dff2d" exitCode=0 Jul 29 21:29:05.770179 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.770008 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq" event={"ID":"cf5d4e51-850c-428f-ac8b-62dbfb1516fb","Type":"ContainerDied","Data":"04902041b93edb199fc1c55d73c3bccce0775da4bde743e2e90815e2668dff2d"} Jul 29 21:29:05.772026 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.771915 2559 generic.go:356] "Generic (PLEG): container finished" podID="97a43b3e-d3c3-41b7-b1d1-349f5d1d7042" containerID="168902d95cf2960fb0830f06a446b95c637eaa515565be5068d5480658790cee" exitCode=0 Jul 29 21:29:05.772026 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.771956 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz" event={"ID":"97a43b3e-d3c3-41b7-b1d1-349f5d1d7042","Type":"ContainerDied","Data":"168902d95cf2960fb0830f06a446b95c637eaa515565be5068d5480658790cee"} Jul 29 21:29:05.860153 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.860128 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz" Jul 29 21:29:05.863523 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.863494 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq" Jul 29 21:29:05.882391 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.882368 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7w4z\" (UniqueName: \"kubernetes.io/projected/cf5d4e51-850c-428f-ac8b-62dbfb1516fb-kube-api-access-p7w4z\") pod \"cf5d4e51-850c-428f-ac8b-62dbfb1516fb\" (UID: \"cf5d4e51-850c-428f-ac8b-62dbfb1516fb\") " Jul 29 21:29:05.882491 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.882397 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blgsn\" (UniqueName: \"kubernetes.io/projected/97a43b3e-d3c3-41b7-b1d1-349f5d1d7042-kube-api-access-blgsn\") pod \"97a43b3e-d3c3-41b7-b1d1-349f5d1d7042\" (UID: \"97a43b3e-d3c3-41b7-b1d1-349f5d1d7042\") " Jul 29 21:29:05.882491 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.882475 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf5d4e51-850c-428f-ac8b-62dbfb1516fb-plugins-config-volume\") pod \"cf5d4e51-850c-428f-ac8b-62dbfb1516fb\" (UID: \"cf5d4e51-850c-428f-ac8b-62dbfb1516fb\") " Jul 29 21:29:05.882569 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.882517 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97a43b3e-d3c3-41b7-b1d1-349f5d1d7042-plugins-config-volume\") pod \"97a43b3e-d3c3-41b7-b1d1-349f5d1d7042\" (UID: \"97a43b3e-d3c3-41b7-b1d1-349f5d1d7042\") " Jul 29 21:29:05.882861 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.882838 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf5d4e51-850c-428f-ac8b-62dbfb1516fb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cf5d4e51-850c-428f-ac8b-62dbfb1516fb" (UID: "cf5d4e51-850c-428f-ac8b-62dbfb1516fb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:29:05.882961 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.882898 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97a43b3e-d3c3-41b7-b1d1-349f5d1d7042-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "97a43b3e-d3c3-41b7-b1d1-349f5d1d7042" (UID: "97a43b3e-d3c3-41b7-b1d1-349f5d1d7042"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:29:05.884488 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.884461 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf5d4e51-850c-428f-ac8b-62dbfb1516fb-kube-api-access-p7w4z" (OuterVolumeSpecName: "kube-api-access-p7w4z") pod "cf5d4e51-850c-428f-ac8b-62dbfb1516fb" (UID: "cf5d4e51-850c-428f-ac8b-62dbfb1516fb"). InnerVolumeSpecName "kube-api-access-p7w4z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:29:05.884488 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.884474 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97a43b3e-d3c3-41b7-b1d1-349f5d1d7042-kube-api-access-blgsn" (OuterVolumeSpecName: "kube-api-access-blgsn") pod "97a43b3e-d3c3-41b7-b1d1-349f5d1d7042" (UID: "97a43b3e-d3c3-41b7-b1d1-349f5d1d7042"). InnerVolumeSpecName "kube-api-access-blgsn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:29:05.983206 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.983168 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blgsn\" (UniqueName: \"kubernetes.io/projected/97a43b3e-d3c3-41b7-b1d1-349f5d1d7042-kube-api-access-blgsn\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:29:05.983206 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.983198 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf5d4e51-850c-428f-ac8b-62dbfb1516fb-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:29:05.983206 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.983208 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97a43b3e-d3c3-41b7-b1d1-349f5d1d7042-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:29:05.983439 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:05.983216 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7w4z\" (UniqueName: \"kubernetes.io/projected/cf5d4e51-850c-428f-ac8b-62dbfb1516fb-kube-api-access-p7w4z\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:29:06.778010 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:06.777979 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq" Jul 29 21:29:06.778499 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:06.778005 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq" event={"ID":"cf5d4e51-850c-428f-ac8b-62dbfb1516fb","Type":"ContainerDied","Data":"051a658458e3b51f1c5c903b1404797cea1ef989331abd6e0f36ebc63e0adcf0"} Jul 29 21:29:06.778499 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:06.778100 2559 scope.go:117] "RemoveContainer" containerID="04902041b93edb199fc1c55d73c3bccce0775da4bde743e2e90815e2668dff2d" Jul 29 21:29:06.779528 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:06.779509 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz" Jul 29 21:29:06.779630 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:06.779528 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz" event={"ID":"97a43b3e-d3c3-41b7-b1d1-349f5d1d7042","Type":"ContainerDied","Data":"4b87a941b0df1b82504ff60df4f8a5240e7fb7a47b91d5fc2e6c735a9eb9e53a"} Jul 29 21:29:06.787812 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:06.787790 2559 scope.go:117] "RemoveContainer" containerID="168902d95cf2960fb0830f06a446b95c637eaa515565be5068d5480658790cee" Jul 29 21:29:06.804212 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:06.804187 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq"] Jul 29 21:29:06.808669 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:06.808648 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-46f813-6f746b4596-84kzq"] Jul 29 21:29:06.824629 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:06.820391 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz"] Jul 29 21:29:06.826790 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:06.826771 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-46f813-6cb6dbd4d4-qllxz"] Jul 29 21:29:07.024939 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:07.024900 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97a43b3e-d3c3-41b7-b1d1-349f5d1d7042" path="/var/lib/kubelet/pods/97a43b3e-d3c3-41b7-b1d1-349f5d1d7042/volumes" Jul 29 21:29:07.025379 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:07.025360 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf5d4e51-850c-428f-ac8b-62dbfb1516fb" path="/var/lib/kubelet/pods/cf5d4e51-850c-428f-ac8b-62dbfb1516fb/volumes" Jul 29 21:29:08.442649 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:08.442617 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:29:16.825790 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.825758 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn"] Jul 29 21:29:16.826307 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.826281 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8e97229-9126-442f-a329-2682c40d9d58" containerName="revoke-keys" Jul 29 21:29:16.826307 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.826306 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8e97229-9126-442f-a329-2682c40d9d58" containerName="revoke-keys" Jul 29 21:29:16.826435 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.826326 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b38e3b4-056f-4515-ae79-705ee6e0508a" containerName="payload-processing" Jul 29 21:29:16.826435 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.826335 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b38e3b4-056f-4515-ae79-705ee6e0508a" containerName="payload-processing" Jul 29 21:29:16.826435 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.826357 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf5d4e51-850c-428f-ac8b-62dbfb1516fb" containerName="payload-pre-processing" Jul 29 21:29:16.826435 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.826367 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf5d4e51-850c-428f-ac8b-62dbfb1516fb" containerName="payload-pre-processing" Jul 29 21:29:16.826435 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.826381 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97a43b3e-d3c3-41b7-b1d1-349f5d1d7042" containerName="payload-processing" Jul 29 21:29:16.826435 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.826386 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="97a43b3e-d3c3-41b7-b1d1-349f5d1d7042" containerName="payload-processing" Jul 29 21:29:16.826705 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.826485 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b38e3b4-056f-4515-ae79-705ee6e0508a" containerName="payload-processing" Jul 29 21:29:16.826705 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.826498 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf5d4e51-850c-428f-ac8b-62dbfb1516fb" containerName="payload-pre-processing" Jul 29 21:29:16.826705 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.826508 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8e97229-9126-442f-a329-2682c40d9d58" containerName="revoke-keys" Jul 29 21:29:16.826705 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.826516 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="97a43b3e-d3c3-41b7-b1d1-349f5d1d7042" containerName="payload-processing" Jul 29 21:29:16.830897 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.830880 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn" Jul 29 21:29:16.833435 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.833417 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-0109d4\"" Jul 29 21:29:16.837949 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.837897 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn"] Jul 29 21:29:16.857246 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.857222 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k"] Jul 29 21:29:16.860392 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.860379 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k" Jul 29 21:29:16.863213 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.863194 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-0109d4-dockercfg-wg4tf\"" Jul 29 21:29:16.874883 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.874861 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k"] Jul 29 21:29:16.877213 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.877187 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdbhg\" (UniqueName: \"kubernetes.io/projected/5caca0f0-019e-4a1f-b91e-9290a452b796-kube-api-access-zdbhg\") pod \"payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn\" (UID: \"5caca0f0-019e-4a1f-b91e-9290a452b796\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn" Jul 29 21:29:16.877325 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.877252 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5caca0f0-019e-4a1f-b91e-9290a452b796-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn\" (UID: \"5caca0f0-019e-4a1f-b91e-9290a452b796\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn" Jul 29 21:29:16.978355 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.978324 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rr79\" (UniqueName: \"kubernetes.io/projected/3f22604e-3b22-45ac-8505-fde9e3fbcaab-kube-api-access-5rr79\") pod \"payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k\" (UID: \"3f22604e-3b22-45ac-8505-fde9e3fbcaab\") " pod="openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k" Jul 29 21:29:16.978520 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.978376 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f22604e-3b22-45ac-8505-fde9e3fbcaab-plugins-config-volume\") pod \"payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k\" (UID: \"3f22604e-3b22-45ac-8505-fde9e3fbcaab\") " pod="openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k" Jul 29 21:29:16.978520 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.978446 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdbhg\" (UniqueName: \"kubernetes.io/projected/5caca0f0-019e-4a1f-b91e-9290a452b796-kube-api-access-zdbhg\") pod \"payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn\" (UID: \"5caca0f0-019e-4a1f-b91e-9290a452b796\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn" Jul 29 21:29:16.978520 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.978498 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5caca0f0-019e-4a1f-b91e-9290a452b796-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn\" (UID: \"5caca0f0-019e-4a1f-b91e-9290a452b796\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn" Jul 29 21:29:16.979179 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.979162 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5caca0f0-019e-4a1f-b91e-9290a452b796-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn\" (UID: \"5caca0f0-019e-4a1f-b91e-9290a452b796\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn" Jul 29 21:29:16.987260 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:16.987237 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdbhg\" (UniqueName: \"kubernetes.io/projected/5caca0f0-019e-4a1f-b91e-9290a452b796-kube-api-access-zdbhg\") pod \"payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn\" (UID: \"5caca0f0-019e-4a1f-b91e-9290a452b796\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn" Jul 29 21:29:17.079520 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:17.079441 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f22604e-3b22-45ac-8505-fde9e3fbcaab-plugins-config-volume\") pod \"payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k\" (UID: \"3f22604e-3b22-45ac-8505-fde9e3fbcaab\") " pod="openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k" Jul 29 21:29:17.079660 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:17.079521 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rr79\" (UniqueName: \"kubernetes.io/projected/3f22604e-3b22-45ac-8505-fde9e3fbcaab-kube-api-access-5rr79\") pod \"payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k\" (UID: \"3f22604e-3b22-45ac-8505-fde9e3fbcaab\") " pod="openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k" Jul 29 21:29:17.080150 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:17.080131 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f22604e-3b22-45ac-8505-fde9e3fbcaab-plugins-config-volume\") pod \"payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k\" (UID: \"3f22604e-3b22-45ac-8505-fde9e3fbcaab\") " pod="openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k" Jul 29 21:29:17.094715 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:17.094688 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rr79\" (UniqueName: \"kubernetes.io/projected/3f22604e-3b22-45ac-8505-fde9e3fbcaab-kube-api-access-5rr79\") pod \"payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k\" (UID: \"3f22604e-3b22-45ac-8505-fde9e3fbcaab\") " pod="openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k" Jul 29 21:29:17.141481 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:17.141440 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn" Jul 29 21:29:17.169471 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:17.169445 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k" Jul 29 21:29:17.312262 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:17.312228 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn"] Jul 29 21:29:17.313728 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:29:17.313703 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5caca0f0_019e_4a1f_b91e_9290a452b796.slice/crio-2260a076bc5da79171350b8e5e6c57debca39aa5dd31d3a1eca4ca2c61206b7f WatchSource:0}: Error finding container 2260a076bc5da79171350b8e5e6c57debca39aa5dd31d3a1eca4ca2c61206b7f: Status 404 returned error can't find the container with id 2260a076bc5da79171350b8e5e6c57debca39aa5dd31d3a1eca4ca2c61206b7f Jul 29 21:29:17.340221 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:17.340199 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k"] Jul 29 21:29:17.350361 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:29:17.350333 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f22604e_3b22_45ac_8505_fde9e3fbcaab.slice/crio-51f2b7f4d685f46f50bac9f0ce47151b3bcc8649433378b99807d2775ff86006 WatchSource:0}: Error finding container 51f2b7f4d685f46f50bac9f0ce47151b3bcc8649433378b99807d2775ff86006: Status 404 returned error can't find the container with id 51f2b7f4d685f46f50bac9f0ce47151b3bcc8649433378b99807d2775ff86006 Jul 29 21:29:17.834301 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:17.834272 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k" event={"ID":"3f22604e-3b22-45ac-8505-fde9e3fbcaab","Type":"ContainerStarted","Data":"10aa0ed8d2bd5b0c9f6ee3428afe43bf2a1b994340008c9490d3789e2693937f"} Jul 29 21:29:17.834787 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:17.834325 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k" event={"ID":"3f22604e-3b22-45ac-8505-fde9e3fbcaab","Type":"ContainerStarted","Data":"51f2b7f4d685f46f50bac9f0ce47151b3bcc8649433378b99807d2775ff86006"} Jul 29 21:29:17.834787 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:17.834360 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k" Jul 29 21:29:17.835704 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:17.835680 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn" event={"ID":"5caca0f0-019e-4a1f-b91e-9290a452b796","Type":"ContainerStarted","Data":"be6232e3abec24a8db508e9880e5c3a5db20649e011672b9c8f02bfc0a6885bc"} Jul 29 21:29:17.835828 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:17.835720 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn" event={"ID":"5caca0f0-019e-4a1f-b91e-9290a452b796","Type":"ContainerStarted","Data":"2260a076bc5da79171350b8e5e6c57debca39aa5dd31d3a1eca4ca2c61206b7f"} Jul 29 21:29:17.835828 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:17.835756 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn" Jul 29 21:29:17.852333 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:17.852282 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k" podStartSLOduration=1.852267712 podStartE2EDuration="1.852267712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:29:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:29:17.851109434 +0000 UTC m=+3853.366415658" watchObservedRunningTime="2026-07-29 21:29:17.852267712 +0000 UTC m=+3853.367573935" Jul 29 21:29:17.871963 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:17.871924 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn" podStartSLOduration=1.8719129159999999 podStartE2EDuration="1.871912916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:29:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:29:17.869790184 +0000 UTC m=+3853.385096432" watchObservedRunningTime="2026-07-29 21:29:17.871912916 +0000 UTC m=+3853.387219140" Jul 29 21:29:22.536710 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:22.536675 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:29:28.843179 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:28.843086 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k" Jul 29 21:29:28.843179 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:28.843177 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn" Jul 29 21:29:46.271886 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.271850 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf"] Jul 29 21:29:46.275427 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.275410 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf" Jul 29 21:29:46.278295 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.278272 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-dfd5ca\"" Jul 29 21:29:46.280748 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.280728 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg"] Jul 29 21:29:46.284153 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.284135 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg" Jul 29 21:29:46.286417 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.286396 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-dfd5ca-dockercfg-x9jn7\"" Jul 29 21:29:46.303968 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.303930 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf"] Jul 29 21:29:46.306407 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.306385 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg"] Jul 29 21:29:46.341180 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.341151 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4ec390e-eaf5-4575-9065-f904c57bc94a-plugins-config-volume\") pod \"payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg\" (UID: \"b4ec390e-eaf5-4575-9065-f904c57bc94a\") " pod="openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg" Jul 29 21:29:46.341180 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.341184 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnh5c\" (UniqueName: \"kubernetes.io/projected/c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5-kube-api-access-xnh5c\") pod \"payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf\" (UID: \"c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf" Jul 29 21:29:46.341398 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.341271 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvwmd\" (UniqueName: \"kubernetes.io/projected/b4ec390e-eaf5-4575-9065-f904c57bc94a-kube-api-access-rvwmd\") pod \"payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg\" (UID: \"b4ec390e-eaf5-4575-9065-f904c57bc94a\") " pod="openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg" Jul 29 21:29:46.341398 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.341304 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf\" (UID: \"c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf" Jul 29 21:29:46.442398 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.442357 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf\" (UID: \"c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf" Jul 29 21:29:46.442579 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.442413 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4ec390e-eaf5-4575-9065-f904c57bc94a-plugins-config-volume\") pod \"payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg\" (UID: \"b4ec390e-eaf5-4575-9065-f904c57bc94a\") " pod="openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg" Jul 29 21:29:46.442579 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.442432 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnh5c\" (UniqueName: \"kubernetes.io/projected/c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5-kube-api-access-xnh5c\") pod \"payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf\" (UID: \"c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf" Jul 29 21:29:46.442579 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.442489 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvwmd\" (UniqueName: \"kubernetes.io/projected/b4ec390e-eaf5-4575-9065-f904c57bc94a-kube-api-access-rvwmd\") pod \"payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg\" (UID: \"b4ec390e-eaf5-4575-9065-f904c57bc94a\") " pod="openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg" Jul 29 21:29:46.443341 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.443317 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf\" (UID: \"c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf" Jul 29 21:29:46.443415 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.443317 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4ec390e-eaf5-4575-9065-f904c57bc94a-plugins-config-volume\") pod \"payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg\" (UID: \"b4ec390e-eaf5-4575-9065-f904c57bc94a\") " pod="openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg" Jul 29 21:29:46.451332 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.451309 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvwmd\" (UniqueName: \"kubernetes.io/projected/b4ec390e-eaf5-4575-9065-f904c57bc94a-kube-api-access-rvwmd\") pod \"payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg\" (UID: \"b4ec390e-eaf5-4575-9065-f904c57bc94a\") " pod="openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg" Jul 29 21:29:46.451454 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.451374 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnh5c\" (UniqueName: \"kubernetes.io/projected/c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5-kube-api-access-xnh5c\") pod \"payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf\" (UID: \"c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf" Jul 29 21:29:46.586191 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.586099 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf" Jul 29 21:29:46.593387 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.593369 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg" Jul 29 21:29:46.729835 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.729704 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf"] Jul 29 21:29:46.732742 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:29:46.732711 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0621cf8_baaa_4cc8_b91e_3e319fe6e3e5.slice/crio-841000c3e3f985cdde1fe660a1a6af84e83f42a7cd1198edb675fa0ac0c96b7b WatchSource:0}: Error finding container 841000c3e3f985cdde1fe660a1a6af84e83f42a7cd1198edb675fa0ac0c96b7b: Status 404 returned error can't find the container with id 841000c3e3f985cdde1fe660a1a6af84e83f42a7cd1198edb675fa0ac0c96b7b Jul 29 21:29:46.754539 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.754520 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg"] Jul 29 21:29:46.765002 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:29:46.764981 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4ec390e_eaf5_4575_9065_f904c57bc94a.slice/crio-fdd66e2cfb13321569ffa92dca4ef80e5f95d550149badaad8f1312399aebc07 WatchSource:0}: Error finding container fdd66e2cfb13321569ffa92dca4ef80e5f95d550149badaad8f1312399aebc07: Status 404 returned error can't find the container with id fdd66e2cfb13321569ffa92dca4ef80e5f95d550149badaad8f1312399aebc07 Jul 29 21:29:46.970542 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.970439 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg" event={"ID":"b4ec390e-eaf5-4575-9065-f904c57bc94a","Type":"ContainerStarted","Data":"c495a7184d4e13b4e318f7277a30ec7e9d42c530314ab9206d886c4f0c81b77b"} Jul 29 21:29:46.970542 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.970501 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg" event={"ID":"b4ec390e-eaf5-4575-9065-f904c57bc94a","Type":"ContainerStarted","Data":"fdd66e2cfb13321569ffa92dca4ef80e5f95d550149badaad8f1312399aebc07"} Jul 29 21:29:46.970778 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.970594 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg" Jul 29 21:29:46.971911 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.971881 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf" event={"ID":"c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5","Type":"ContainerStarted","Data":"98ce147275527ee8b547773c1de0a9978791fc30baac05e46fc4b7502d844915"} Jul 29 21:29:46.972010 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.971932 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf" Jul 29 21:29:46.972010 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.971950 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf" event={"ID":"c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5","Type":"ContainerStarted","Data":"841000c3e3f985cdde1fe660a1a6af84e83f42a7cd1198edb675fa0ac0c96b7b"} Jul 29 21:29:46.990901 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:46.990857 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg" podStartSLOduration=0.990841703 podStartE2EDuration="990.841703ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:29:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:29:46.989424362 +0000 UTC m=+3882.504730586" watchObservedRunningTime="2026-07-29 21:29:46.990841703 +0000 UTC m=+3882.506147956" Jul 29 21:29:47.012918 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:47.012853 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf" podStartSLOduration=1.012831807 podStartE2EDuration="1.012831807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:29:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:29:47.010485588 +0000 UTC m=+3882.525791824" watchObservedRunningTime="2026-07-29 21:29:47.012831807 +0000 UTC m=+3882.528138033" Jul 29 21:29:52.233481 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:52.233441 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:29:57.978528 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:57.978499 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg" Jul 29 21:29:57.978997 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:29:57.978613 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf" Jul 29 21:30:25.705291 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:25.705254 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k"] Jul 29 21:30:25.707903 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:25.705486 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k" podUID="3f22604e-3b22-45ac-8505-fde9e3fbcaab" containerName="payload-processing" containerID="cri-o://10aa0ed8d2bd5b0c9f6ee3428afe43bf2a1b994340008c9490d3789e2693937f" gracePeriod=30 Jul 29 21:30:25.711875 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:25.711849 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn"] Jul 29 21:30:25.712163 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:25.712104 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn" podUID="5caca0f0-019e-4a1f-b91e-9290a452b796" containerName="payload-pre-processing" containerID="cri-o://be6232e3abec24a8db508e9880e5c3a5db20649e011672b9c8f02bfc0a6885bc" gracePeriod=30 Jul 29 21:30:25.980743 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:25.980718 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k" Jul 29 21:30:25.983467 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:25.983444 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn" Jul 29 21:30:26.069696 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.069663 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdbhg\" (UniqueName: \"kubernetes.io/projected/5caca0f0-019e-4a1f-b91e-9290a452b796-kube-api-access-zdbhg\") pod \"5caca0f0-019e-4a1f-b91e-9290a452b796\" (UID: \"5caca0f0-019e-4a1f-b91e-9290a452b796\") " Jul 29 21:30:26.069696 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.069698 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rr79\" (UniqueName: \"kubernetes.io/projected/3f22604e-3b22-45ac-8505-fde9e3fbcaab-kube-api-access-5rr79\") pod \"3f22604e-3b22-45ac-8505-fde9e3fbcaab\" (UID: \"3f22604e-3b22-45ac-8505-fde9e3fbcaab\") " Jul 29 21:30:26.069947 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.069724 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5caca0f0-019e-4a1f-b91e-9290a452b796-plugins-config-volume\") pod \"5caca0f0-019e-4a1f-b91e-9290a452b796\" (UID: \"5caca0f0-019e-4a1f-b91e-9290a452b796\") " Jul 29 21:30:26.069947 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.069748 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f22604e-3b22-45ac-8505-fde9e3fbcaab-plugins-config-volume\") pod \"3f22604e-3b22-45ac-8505-fde9e3fbcaab\" (UID: \"3f22604e-3b22-45ac-8505-fde9e3fbcaab\") " Jul 29 21:30:26.070156 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.070081 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5caca0f0-019e-4a1f-b91e-9290a452b796-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5caca0f0-019e-4a1f-b91e-9290a452b796" (UID: "5caca0f0-019e-4a1f-b91e-9290a452b796"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:30:26.070195 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.070164 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f22604e-3b22-45ac-8505-fde9e3fbcaab-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3f22604e-3b22-45ac-8505-fde9e3fbcaab" (UID: "3f22604e-3b22-45ac-8505-fde9e3fbcaab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:30:26.071734 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.071713 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f22604e-3b22-45ac-8505-fde9e3fbcaab-kube-api-access-5rr79" (OuterVolumeSpecName: "kube-api-access-5rr79") pod "3f22604e-3b22-45ac-8505-fde9e3fbcaab" (UID: "3f22604e-3b22-45ac-8505-fde9e3fbcaab"). InnerVolumeSpecName "kube-api-access-5rr79". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:30:26.071817 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.071731 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5caca0f0-019e-4a1f-b91e-9290a452b796-kube-api-access-zdbhg" (OuterVolumeSpecName: "kube-api-access-zdbhg") pod "5caca0f0-019e-4a1f-b91e-9290a452b796" (UID: "5caca0f0-019e-4a1f-b91e-9290a452b796"). InnerVolumeSpecName "kube-api-access-zdbhg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:30:26.159036 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.159005 2559 generic.go:356] "Generic (PLEG): container finished" podID="3f22604e-3b22-45ac-8505-fde9e3fbcaab" containerID="10aa0ed8d2bd5b0c9f6ee3428afe43bf2a1b994340008c9490d3789e2693937f" exitCode=0 Jul 29 21:30:26.159175 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.159062 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k" Jul 29 21:30:26.159175 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.159105 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k" event={"ID":"3f22604e-3b22-45ac-8505-fde9e3fbcaab","Type":"ContainerDied","Data":"10aa0ed8d2bd5b0c9f6ee3428afe43bf2a1b994340008c9490d3789e2693937f"} Jul 29 21:30:26.159175 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.159153 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k" event={"ID":"3f22604e-3b22-45ac-8505-fde9e3fbcaab","Type":"ContainerDied","Data":"51f2b7f4d685f46f50bac9f0ce47151b3bcc8649433378b99807d2775ff86006"} Jul 29 21:30:26.159175 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.159176 2559 scope.go:117] "RemoveContainer" containerID="10aa0ed8d2bd5b0c9f6ee3428afe43bf2a1b994340008c9490d3789e2693937f" Jul 29 21:30:26.160365 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.160343 2559 generic.go:356] "Generic (PLEG): container finished" podID="5caca0f0-019e-4a1f-b91e-9290a452b796" containerID="be6232e3abec24a8db508e9880e5c3a5db20649e011672b9c8f02bfc0a6885bc" exitCode=0 Jul 29 21:30:26.160449 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.160386 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn" Jul 29 21:30:26.160449 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.160426 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn" event={"ID":"5caca0f0-019e-4a1f-b91e-9290a452b796","Type":"ContainerDied","Data":"be6232e3abec24a8db508e9880e5c3a5db20649e011672b9c8f02bfc0a6885bc"} Jul 29 21:30:26.160551 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.160471 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn" event={"ID":"5caca0f0-019e-4a1f-b91e-9290a452b796","Type":"ContainerDied","Data":"2260a076bc5da79171350b8e5e6c57debca39aa5dd31d3a1eca4ca2c61206b7f"} Jul 29 21:30:26.168250 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.168120 2559 scope.go:117] "RemoveContainer" containerID="10aa0ed8d2bd5b0c9f6ee3428afe43bf2a1b994340008c9490d3789e2693937f" Jul 29 21:30:26.168436 ip-10-0-138-67 kubenswrapper[2559]: E0729 21:30:26.168412 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10aa0ed8d2bd5b0c9f6ee3428afe43bf2a1b994340008c9490d3789e2693937f\": container with ID starting with 10aa0ed8d2bd5b0c9f6ee3428afe43bf2a1b994340008c9490d3789e2693937f not found: ID does not exist" containerID="10aa0ed8d2bd5b0c9f6ee3428afe43bf2a1b994340008c9490d3789e2693937f" Jul 29 21:30:26.168516 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.168445 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10aa0ed8d2bd5b0c9f6ee3428afe43bf2a1b994340008c9490d3789e2693937f"} err="failed to get container status \"10aa0ed8d2bd5b0c9f6ee3428afe43bf2a1b994340008c9490d3789e2693937f\": rpc error: code = NotFound desc = could not find container \"10aa0ed8d2bd5b0c9f6ee3428afe43bf2a1b994340008c9490d3789e2693937f\": container with ID starting with 10aa0ed8d2bd5b0c9f6ee3428afe43bf2a1b994340008c9490d3789e2693937f not found: ID does not exist" Jul 29 21:30:26.168516 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.168468 2559 scope.go:117] "RemoveContainer" containerID="be6232e3abec24a8db508e9880e5c3a5db20649e011672b9c8f02bfc0a6885bc" Jul 29 21:30:26.170385 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.170361 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdbhg\" (UniqueName: \"kubernetes.io/projected/5caca0f0-019e-4a1f-b91e-9290a452b796-kube-api-access-zdbhg\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:30:26.170470 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.170391 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rr79\" (UniqueName: \"kubernetes.io/projected/3f22604e-3b22-45ac-8505-fde9e3fbcaab-kube-api-access-5rr79\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:30:26.170470 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.170408 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5caca0f0-019e-4a1f-b91e-9290a452b796-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:30:26.170470 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.170422 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f22604e-3b22-45ac-8505-fde9e3fbcaab-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:30:26.175889 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.175874 2559 scope.go:117] "RemoveContainer" containerID="be6232e3abec24a8db508e9880e5c3a5db20649e011672b9c8f02bfc0a6885bc" Jul 29 21:30:26.176134 ip-10-0-138-67 kubenswrapper[2559]: E0729 21:30:26.176118 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be6232e3abec24a8db508e9880e5c3a5db20649e011672b9c8f02bfc0a6885bc\": container with ID starting with be6232e3abec24a8db508e9880e5c3a5db20649e011672b9c8f02bfc0a6885bc not found: ID does not exist" containerID="be6232e3abec24a8db508e9880e5c3a5db20649e011672b9c8f02bfc0a6885bc" Jul 29 21:30:26.176180 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.176140 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be6232e3abec24a8db508e9880e5c3a5db20649e011672b9c8f02bfc0a6885bc"} err="failed to get container status \"be6232e3abec24a8db508e9880e5c3a5db20649e011672b9c8f02bfc0a6885bc\": rpc error: code = NotFound desc = could not find container \"be6232e3abec24a8db508e9880e5c3a5db20649e011672b9c8f02bfc0a6885bc\": container with ID starting with be6232e3abec24a8db508e9880e5c3a5db20649e011672b9c8f02bfc0a6885bc not found: ID does not exist" Jul 29 21:30:26.184923 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.184902 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn"] Jul 29 21:30:26.193018 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.192992 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0109d4-7c9b877744-dw9xn"] Jul 29 21:30:26.206133 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.206100 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k"] Jul 29 21:30:26.208123 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:26.208073 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-0109d4-8b8b95776-jnd8k"] Jul 29 21:30:27.025021 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:27.024991 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f22604e-3b22-45ac-8505-fde9e3fbcaab" path="/var/lib/kubelet/pods/3f22604e-3b22-45ac-8505-fde9e3fbcaab/volumes" Jul 29 21:30:27.025407 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:27.025352 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5caca0f0-019e-4a1f-b91e-9290a452b796" path="/var/lib/kubelet/pods/5caca0f0-019e-4a1f-b91e-9290a452b796/volumes" Jul 29 21:30:31.043186 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:31.043150 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:30:43.950299 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:43.950266 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg"] Jul 29 21:30:43.950740 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:43.950510 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg" podUID="b4ec390e-eaf5-4575-9065-f904c57bc94a" containerName="payload-processing" containerID="cri-o://c495a7184d4e13b4e318f7277a30ec7e9d42c530314ab9206d886c4f0c81b77b" gracePeriod=30 Jul 29 21:30:43.958612 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:43.958587 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf"] Jul 29 21:30:43.958819 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:43.958777 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf" podUID="c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5" containerName="payload-pre-processing" containerID="cri-o://98ce147275527ee8b547773c1de0a9978791fc30baac05e46fc4b7502d844915" gracePeriod=30 Jul 29 21:30:44.234787 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.234765 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg" Jul 29 21:30:44.237506 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.237488 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf" Jul 29 21:30:44.245801 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.245774 2559 generic.go:356] "Generic (PLEG): container finished" podID="b4ec390e-eaf5-4575-9065-f904c57bc94a" containerID="c495a7184d4e13b4e318f7277a30ec7e9d42c530314ab9206d886c4f0c81b77b" exitCode=0 Jul 29 21:30:44.245896 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.245814 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg" Jul 29 21:30:44.245896 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.245824 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg" event={"ID":"b4ec390e-eaf5-4575-9065-f904c57bc94a","Type":"ContainerDied","Data":"c495a7184d4e13b4e318f7277a30ec7e9d42c530314ab9206d886c4f0c81b77b"} Jul 29 21:30:44.246024 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.245906 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg" event={"ID":"b4ec390e-eaf5-4575-9065-f904c57bc94a","Type":"ContainerDied","Data":"fdd66e2cfb13321569ffa92dca4ef80e5f95d550149badaad8f1312399aebc07"} Jul 29 21:30:44.246024 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.245938 2559 scope.go:117] "RemoveContainer" containerID="c495a7184d4e13b4e318f7277a30ec7e9d42c530314ab9206d886c4f0c81b77b" Jul 29 21:30:44.247109 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.247086 2559 generic.go:356] "Generic (PLEG): container finished" podID="c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5" containerID="98ce147275527ee8b547773c1de0a9978791fc30baac05e46fc4b7502d844915" exitCode=0 Jul 29 21:30:44.247243 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.247123 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf" Jul 29 21:30:44.247243 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.247169 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf" event={"ID":"c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5","Type":"ContainerDied","Data":"98ce147275527ee8b547773c1de0a9978791fc30baac05e46fc4b7502d844915"} Jul 29 21:30:44.247243 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.247218 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf" event={"ID":"c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5","Type":"ContainerDied","Data":"841000c3e3f985cdde1fe660a1a6af84e83f42a7cd1198edb675fa0ac0c96b7b"} Jul 29 21:30:44.254148 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.254126 2559 scope.go:117] "RemoveContainer" containerID="c495a7184d4e13b4e318f7277a30ec7e9d42c530314ab9206d886c4f0c81b77b" Jul 29 21:30:44.254461 ip-10-0-138-67 kubenswrapper[2559]: E0729 21:30:44.254438 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c495a7184d4e13b4e318f7277a30ec7e9d42c530314ab9206d886c4f0c81b77b\": container with ID starting with c495a7184d4e13b4e318f7277a30ec7e9d42c530314ab9206d886c4f0c81b77b not found: ID does not exist" containerID="c495a7184d4e13b4e318f7277a30ec7e9d42c530314ab9206d886c4f0c81b77b" Jul 29 21:30:44.254540 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.254473 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c495a7184d4e13b4e318f7277a30ec7e9d42c530314ab9206d886c4f0c81b77b"} err="failed to get container status \"c495a7184d4e13b4e318f7277a30ec7e9d42c530314ab9206d886c4f0c81b77b\": rpc error: code = NotFound desc = could not find container \"c495a7184d4e13b4e318f7277a30ec7e9d42c530314ab9206d886c4f0c81b77b\": container with ID starting with c495a7184d4e13b4e318f7277a30ec7e9d42c530314ab9206d886c4f0c81b77b not found: ID does not exist" Jul 29 21:30:44.254540 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.254497 2559 scope.go:117] "RemoveContainer" containerID="98ce147275527ee8b547773c1de0a9978791fc30baac05e46fc4b7502d844915" Jul 29 21:30:44.263144 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.263132 2559 scope.go:117] "RemoveContainer" containerID="98ce147275527ee8b547773c1de0a9978791fc30baac05e46fc4b7502d844915" Jul 29 21:30:44.263383 ip-10-0-138-67 kubenswrapper[2559]: E0729 21:30:44.263365 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98ce147275527ee8b547773c1de0a9978791fc30baac05e46fc4b7502d844915\": container with ID starting with 98ce147275527ee8b547773c1de0a9978791fc30baac05e46fc4b7502d844915 not found: ID does not exist" containerID="98ce147275527ee8b547773c1de0a9978791fc30baac05e46fc4b7502d844915" Jul 29 21:30:44.263462 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.263389 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98ce147275527ee8b547773c1de0a9978791fc30baac05e46fc4b7502d844915"} err="failed to get container status \"98ce147275527ee8b547773c1de0a9978791fc30baac05e46fc4b7502d844915\": rpc error: code = NotFound desc = could not find container \"98ce147275527ee8b547773c1de0a9978791fc30baac05e46fc4b7502d844915\": container with ID starting with 98ce147275527ee8b547773c1de0a9978791fc30baac05e46fc4b7502d844915 not found: ID does not exist" Jul 29 21:30:44.333184 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.333148 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvwmd\" (UniqueName: \"kubernetes.io/projected/b4ec390e-eaf5-4575-9065-f904c57bc94a-kube-api-access-rvwmd\") pod \"b4ec390e-eaf5-4575-9065-f904c57bc94a\" (UID: \"b4ec390e-eaf5-4575-9065-f904c57bc94a\") " Jul 29 21:30:44.333367 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.333215 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5-plugins-config-volume\") pod \"c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5\" (UID: \"c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5\") " Jul 29 21:30:44.333367 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.333240 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4ec390e-eaf5-4575-9065-f904c57bc94a-plugins-config-volume\") pod \"b4ec390e-eaf5-4575-9065-f904c57bc94a\" (UID: \"b4ec390e-eaf5-4575-9065-f904c57bc94a\") " Jul 29 21:30:44.333367 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.333255 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnh5c\" (UniqueName: \"kubernetes.io/projected/c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5-kube-api-access-xnh5c\") pod \"c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5\" (UID: \"c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5\") " Jul 29 21:30:44.333605 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.333578 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5" (UID: "c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:30:44.333666 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.333624 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4ec390e-eaf5-4575-9065-f904c57bc94a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b4ec390e-eaf5-4575-9065-f904c57bc94a" (UID: "b4ec390e-eaf5-4575-9065-f904c57bc94a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:30:44.335290 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.335270 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5-kube-api-access-xnh5c" (OuterVolumeSpecName: "kube-api-access-xnh5c") pod "c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5" (UID: "c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5"). InnerVolumeSpecName "kube-api-access-xnh5c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:30:44.335365 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.335284 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4ec390e-eaf5-4575-9065-f904c57bc94a-kube-api-access-rvwmd" (OuterVolumeSpecName: "kube-api-access-rvwmd") pod "b4ec390e-eaf5-4575-9065-f904c57bc94a" (UID: "b4ec390e-eaf5-4575-9065-f904c57bc94a"). InnerVolumeSpecName "kube-api-access-rvwmd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:30:44.433938 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.433904 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:30:44.433938 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.433933 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4ec390e-eaf5-4575-9065-f904c57bc94a-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:30:44.433938 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.433943 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnh5c\" (UniqueName: \"kubernetes.io/projected/c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5-kube-api-access-xnh5c\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:30:44.434177 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.433951 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvwmd\" (UniqueName: \"kubernetes.io/projected/b4ec390e-eaf5-4575-9065-f904c57bc94a-kube-api-access-rvwmd\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:30:44.571586 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.571552 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg"] Jul 29 21:30:44.576988 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.576967 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-dfd5ca-8477bcfcb8-xlfvg"] Jul 29 21:30:44.588016 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.587992 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf"] Jul 29 21:30:44.592060 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:44.592026 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-dfd5ca-c6c745956-497zf"] Jul 29 21:30:45.025146 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:45.025116 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4ec390e-eaf5-4575-9065-f904c57bc94a" path="/var/lib/kubelet/pods/b4ec390e-eaf5-4575-9065-f904c57bc94a/volumes" Jul 29 21:30:45.025495 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:45.025472 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5" path="/var/lib/kubelet/pods/c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5/volumes" Jul 29 21:30:47.680284 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:47.680251 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:30:55.490418 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.490386 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd"] Jul 29 21:30:55.490795 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.490703 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5" containerName="payload-pre-processing" Jul 29 21:30:55.490795 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.490714 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5" containerName="payload-pre-processing" Jul 29 21:30:55.490795 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.490724 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5caca0f0-019e-4a1f-b91e-9290a452b796" containerName="payload-pre-processing" Jul 29 21:30:55.490795 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.490731 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5caca0f0-019e-4a1f-b91e-9290a452b796" containerName="payload-pre-processing" Jul 29 21:30:55.490795 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.490737 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f22604e-3b22-45ac-8505-fde9e3fbcaab" containerName="payload-processing" Jul 29 21:30:55.490795 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.490743 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f22604e-3b22-45ac-8505-fde9e3fbcaab" containerName="payload-processing" Jul 29 21:30:55.490795 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.490748 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4ec390e-eaf5-4575-9065-f904c57bc94a" containerName="payload-processing" Jul 29 21:30:55.490795 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.490753 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4ec390e-eaf5-4575-9065-f904c57bc94a" containerName="payload-processing" Jul 29 21:30:55.491036 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.490816 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4ec390e-eaf5-4575-9065-f904c57bc94a" containerName="payload-processing" Jul 29 21:30:55.491036 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.490825 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5caca0f0-019e-4a1f-b91e-9290a452b796" containerName="payload-pre-processing" Jul 29 21:30:55.491036 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.490831 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f22604e-3b22-45ac-8505-fde9e3fbcaab" containerName="payload-processing" Jul 29 21:30:55.491036 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.490839 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0621cf8-baaa-4cc8-b91e-3e319fe6e3e5" containerName="payload-pre-processing" Jul 29 21:30:55.493768 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.493747 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd" Jul 29 21:30:55.496748 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.496728 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-5f7d51-dockercfg-5kq65\"" Jul 29 21:30:55.497215 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.497199 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-5f7d51\"" Jul 29 21:30:55.504587 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.504566 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd"] Jul 29 21:30:55.519061 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.519006 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755"] Jul 29 21:30:55.522635 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.522618 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755" Jul 29 21:30:55.532331 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.532312 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755"] Jul 29 21:30:55.534261 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.534242 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91c2102b-7d59-43fd-94a8-8e9dac0d6562-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd\" (UID: \"91c2102b-7d59-43fd-94a8-8e9dac0d6562\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd" Jul 29 21:30:55.534377 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.534313 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5w9l\" (UniqueName: \"kubernetes.io/projected/91c2102b-7d59-43fd-94a8-8e9dac0d6562-kube-api-access-c5w9l\") pod \"payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd\" (UID: \"91c2102b-7d59-43fd-94a8-8e9dac0d6562\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd" Jul 29 21:30:55.635142 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.635107 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd926dff-0074-48fc-9843-a110ec7e1027-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755\" (UID: \"fd926dff-0074-48fc-9843-a110ec7e1027\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755" Jul 29 21:30:55.635323 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.635162 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5w9l\" (UniqueName: \"kubernetes.io/projected/91c2102b-7d59-43fd-94a8-8e9dac0d6562-kube-api-access-c5w9l\") pod \"payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd\" (UID: \"91c2102b-7d59-43fd-94a8-8e9dac0d6562\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd" Jul 29 21:30:55.635323 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.635201 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91c2102b-7d59-43fd-94a8-8e9dac0d6562-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd\" (UID: \"91c2102b-7d59-43fd-94a8-8e9dac0d6562\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd" Jul 29 21:30:55.635323 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.635226 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrkp5\" (UniqueName: \"kubernetes.io/projected/fd926dff-0074-48fc-9843-a110ec7e1027-kube-api-access-lrkp5\") pod \"payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755\" (UID: \"fd926dff-0074-48fc-9843-a110ec7e1027\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755" Jul 29 21:30:55.635850 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.635830 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91c2102b-7d59-43fd-94a8-8e9dac0d6562-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd\" (UID: \"91c2102b-7d59-43fd-94a8-8e9dac0d6562\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd" Jul 29 21:30:55.644069 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.644023 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5w9l\" (UniqueName: \"kubernetes.io/projected/91c2102b-7d59-43fd-94a8-8e9dac0d6562-kube-api-access-c5w9l\") pod \"payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd\" (UID: \"91c2102b-7d59-43fd-94a8-8e9dac0d6562\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd" Jul 29 21:30:55.736407 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.736369 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd926dff-0074-48fc-9843-a110ec7e1027-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755\" (UID: \"fd926dff-0074-48fc-9843-a110ec7e1027\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755" Jul 29 21:30:55.736610 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.736435 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrkp5\" (UniqueName: \"kubernetes.io/projected/fd926dff-0074-48fc-9843-a110ec7e1027-kube-api-access-lrkp5\") pod \"payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755\" (UID: \"fd926dff-0074-48fc-9843-a110ec7e1027\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755" Jul 29 21:30:55.737090 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.737071 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd926dff-0074-48fc-9843-a110ec7e1027-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755\" (UID: \"fd926dff-0074-48fc-9843-a110ec7e1027\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755" Jul 29 21:30:55.745280 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.745259 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrkp5\" (UniqueName: \"kubernetes.io/projected/fd926dff-0074-48fc-9843-a110ec7e1027-kube-api-access-lrkp5\") pod \"payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755\" (UID: \"fd926dff-0074-48fc-9843-a110ec7e1027\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755" Jul 29 21:30:55.804239 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.804205 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd" Jul 29 21:30:55.833302 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.833264 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755" Jul 29 21:30:55.987790 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:55.987767 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755"] Jul 29 21:30:55.989791 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:30:55.989765 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd926dff_0074_48fc_9843_a110ec7e1027.slice/crio-92621f763d6444135fbf294d73fa8e048350b48776f903679e18686027f363d7 WatchSource:0}: Error finding container 92621f763d6444135fbf294d73fa8e048350b48776f903679e18686027f363d7: Status 404 returned error can't find the container with id 92621f763d6444135fbf294d73fa8e048350b48776f903679e18686027f363d7 Jul 29 21:30:56.162408 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:56.162385 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd"] Jul 29 21:30:56.164820 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:30:56.164797 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91c2102b_7d59_43fd_94a8_8e9dac0d6562.slice/crio-244f6271b4df53f43544a37ed5c8807479d63ceeca7cab8b0b930528686796a5 WatchSource:0}: Error finding container 244f6271b4df53f43544a37ed5c8807479d63ceeca7cab8b0b930528686796a5: Status 404 returned error can't find the container with id 244f6271b4df53f43544a37ed5c8807479d63ceeca7cab8b0b930528686796a5 Jul 29 21:30:56.324705 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:56.324598 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755" event={"ID":"fd926dff-0074-48fc-9843-a110ec7e1027","Type":"ContainerStarted","Data":"ea9df1c7499be1ddc2e33be21319ebde3cabcfb74090a3531055e826604227c1"} Jul 29 21:30:56.324705 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:56.324683 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755" event={"ID":"fd926dff-0074-48fc-9843-a110ec7e1027","Type":"ContainerStarted","Data":"92621f763d6444135fbf294d73fa8e048350b48776f903679e18686027f363d7"} Jul 29 21:30:56.324959 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:56.324714 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755" Jul 29 21:30:56.326081 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:56.326037 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd" event={"ID":"91c2102b-7d59-43fd-94a8-8e9dac0d6562","Type":"ContainerStarted","Data":"7b10a1207ac4e807166153078eb4af4afcffaa71ee33f3ffb3f8748e0e15a177"} Jul 29 21:30:56.326171 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:56.326100 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd" event={"ID":"91c2102b-7d59-43fd-94a8-8e9dac0d6562","Type":"ContainerStarted","Data":"244f6271b4df53f43544a37ed5c8807479d63ceeca7cab8b0b930528686796a5"} Jul 29 21:30:56.326231 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:56.326166 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd" Jul 29 21:30:56.341909 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:56.341775 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755" podStartSLOduration=1.341758806 podStartE2EDuration="1.341758806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:30:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:30:56.340635828 +0000 UTC m=+3951.855942046" watchObservedRunningTime="2026-07-29 21:30:56.341758806 +0000 UTC m=+3951.857065033" Jul 29 21:30:56.362857 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:30:56.362812 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd" podStartSLOduration=1.3627978170000001 podStartE2EDuration="1.362797817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:30:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:30:56.361179745 +0000 UTC m=+3951.876485969" watchObservedRunningTime="2026-07-29 21:30:56.362797817 +0000 UTC m=+3951.878104042" Jul 29 21:31:01.943407 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:01.943326 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:31:07.333159 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:07.333121 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755" Jul 29 21:31:07.333591 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:07.333216 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd" Jul 29 21:31:23.743447 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:23.743410 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7"] Jul 29 21:31:23.746702 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:23.746681 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7" Jul 29 21:31:23.749233 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:23.749206 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-eee583-dockercfg-mmkgj\"" Jul 29 21:31:23.749411 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:23.749391 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-eee583\"" Jul 29 21:31:23.755528 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:23.755502 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7"] Jul 29 21:31:23.778514 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:23.778485 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj"] Jul 29 21:31:23.781759 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:23.781741 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj" Jul 29 21:31:23.792884 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:23.792857 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj"] Jul 29 21:31:23.885040 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:23.885003 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c43741d-a07a-491e-909b-117effce6659-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7\" (UID: \"5c43741d-a07a-491e-909b-117effce6659\") " pod="openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7" Jul 29 21:31:23.885244 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:23.885080 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98ed4417-f510-4574-931c-7807ec4ac780-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj\" (UID: \"98ed4417-f510-4574-931c-7807ec4ac780\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj" Jul 29 21:31:23.885244 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:23.885105 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbdj7\" (UniqueName: \"kubernetes.io/projected/5c43741d-a07a-491e-909b-117effce6659-kube-api-access-mbdj7\") pod \"payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7\" (UID: \"5c43741d-a07a-491e-909b-117effce6659\") " pod="openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7" Jul 29 21:31:23.885244 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:23.885134 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww9kw\" (UniqueName: \"kubernetes.io/projected/98ed4417-f510-4574-931c-7807ec4ac780-kube-api-access-ww9kw\") pod \"payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj\" (UID: \"98ed4417-f510-4574-931c-7807ec4ac780\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj" Jul 29 21:31:23.986447 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:23.986412 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c43741d-a07a-491e-909b-117effce6659-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7\" (UID: \"5c43741d-a07a-491e-909b-117effce6659\") " pod="openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7" Jul 29 21:31:23.986634 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:23.986457 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98ed4417-f510-4574-931c-7807ec4ac780-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj\" (UID: \"98ed4417-f510-4574-931c-7807ec4ac780\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj" Jul 29 21:31:23.986634 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:23.986476 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbdj7\" (UniqueName: \"kubernetes.io/projected/5c43741d-a07a-491e-909b-117effce6659-kube-api-access-mbdj7\") pod \"payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7\" (UID: \"5c43741d-a07a-491e-909b-117effce6659\") " pod="openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7" Jul 29 21:31:23.986634 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:23.986497 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww9kw\" (UniqueName: \"kubernetes.io/projected/98ed4417-f510-4574-931c-7807ec4ac780-kube-api-access-ww9kw\") pod \"payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj\" (UID: \"98ed4417-f510-4574-931c-7807ec4ac780\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj" Jul 29 21:31:23.987181 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:23.987156 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c43741d-a07a-491e-909b-117effce6659-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7\" (UID: \"5c43741d-a07a-491e-909b-117effce6659\") " pod="openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7" Jul 29 21:31:23.987279 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:23.987190 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98ed4417-f510-4574-931c-7807ec4ac780-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj\" (UID: \"98ed4417-f510-4574-931c-7807ec4ac780\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj" Jul 29 21:31:23.993993 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:23.993933 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww9kw\" (UniqueName: \"kubernetes.io/projected/98ed4417-f510-4574-931c-7807ec4ac780-kube-api-access-ww9kw\") pod \"payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj\" (UID: \"98ed4417-f510-4574-931c-7807ec4ac780\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj" Jul 29 21:31:23.994095 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:23.994041 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbdj7\" (UniqueName: \"kubernetes.io/projected/5c43741d-a07a-491e-909b-117effce6659-kube-api-access-mbdj7\") pod \"payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7\" (UID: \"5c43741d-a07a-491e-909b-117effce6659\") " pod="openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7" Jul 29 21:31:24.058619 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:24.058596 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7" Jul 29 21:31:24.091652 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:24.091625 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj" Jul 29 21:31:24.206718 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:24.206685 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7"] Jul 29 21:31:24.207952 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:31:24.207920 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c43741d_a07a_491e_909b_117effce6659.slice/crio-642217fce5a1c8150a8ccb48ea8ebedec264deac3ecb5ab12d64671d5dea6fe6 WatchSource:0}: Error finding container 642217fce5a1c8150a8ccb48ea8ebedec264deac3ecb5ab12d64671d5dea6fe6: Status 404 returned error can't find the container with id 642217fce5a1c8150a8ccb48ea8ebedec264deac3ecb5ab12d64671d5dea6fe6 Jul 29 21:31:24.235285 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:24.235264 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj"] Jul 29 21:31:24.247124 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:31:24.247094 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98ed4417_f510_4574_931c_7807ec4ac780.slice/crio-4f910346fbbea1af2a1a62be2c2d4d367afd6ac6e794c624c071f0d8182e0b21 WatchSource:0}: Error finding container 4f910346fbbea1af2a1a62be2c2d4d367afd6ac6e794c624c071f0d8182e0b21: Status 404 returned error can't find the container with id 4f910346fbbea1af2a1a62be2c2d4d367afd6ac6e794c624c071f0d8182e0b21 Jul 29 21:31:24.467038 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:24.466991 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7" event={"ID":"5c43741d-a07a-491e-909b-117effce6659","Type":"ContainerStarted","Data":"dc734a767e93676cde95b85b853a22c9d75a261380b15dfabe6965e05acbed49"} Jul 29 21:31:24.467225 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:24.467177 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7" Jul 29 21:31:24.467225 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:24.467214 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7" event={"ID":"5c43741d-a07a-491e-909b-117effce6659","Type":"ContainerStarted","Data":"642217fce5a1c8150a8ccb48ea8ebedec264deac3ecb5ab12d64671d5dea6fe6"} Jul 29 21:31:24.468471 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:24.468448 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj" event={"ID":"98ed4417-f510-4574-931c-7807ec4ac780","Type":"ContainerStarted","Data":"3e8487220b6d5b8d100f01ce3865b6a4652632b89c12dcb910caa65c4ff1f704"} Jul 29 21:31:24.468585 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:24.468493 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj" event={"ID":"98ed4417-f510-4574-931c-7807ec4ac780","Type":"ContainerStarted","Data":"4f910346fbbea1af2a1a62be2c2d4d367afd6ac6e794c624c071f0d8182e0b21"} Jul 29 21:31:24.468585 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:24.468518 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj" Jul 29 21:31:24.487851 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:24.487805 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7" podStartSLOduration=1.4877911400000001 podStartE2EDuration="1.48779114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:31:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:31:24.487382707 +0000 UTC m=+3980.002688932" watchObservedRunningTime="2026-07-29 21:31:24.48779114 +0000 UTC m=+3980.003097364" Jul 29 21:31:24.507211 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:24.507129 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj" podStartSLOduration=1.50711095 podStartE2EDuration="1.50711095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:31:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:31:24.507083401 +0000 UTC m=+3980.022389627" watchObservedRunningTime="2026-07-29 21:31:24.50711095 +0000 UTC m=+3980.022417175" Jul 29 21:31:30.130768 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:30.130734 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:31:35.475792 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:35.475757 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj" Jul 29 21:31:35.476311 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:31:35.475836 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7" Jul 29 21:32:05.732102 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:32:05.732071 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:32:22.736750 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:32:22.736709 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:32:48.932940 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:32:48.932847 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:33:14.828963 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:33:14.828921 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:33:34.931480 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:33:34.931443 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:34:01.140388 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:34:01.140309 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:34:20.634085 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:34:20.634034 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:34:47.233418 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:34:47.233376 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:35:05.396931 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:35:05.396896 2559 scope.go:117] "RemoveContainer" containerID="e472a093ee50205fa9a7d87cd36c8768bf5fedc42e26223b919cfdf325eaff59" Jul 29 21:35:06.637215 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:35:06.637179 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:35:33.146945 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:35:33.146848 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:35:50.738509 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:35:50.738476 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:36:17.832899 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:36:17.832847 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:36:35.934496 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:36:35.934463 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:37:03.834118 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:37:03.834019 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:37:21.052456 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:37:21.052413 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:37:33.888949 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:37:33.888908 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:38:13.291140 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:38:13.291096 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:38:59.734665 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:38:59.734567 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:39:35.743599 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:39:35.743560 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:40:16.937724 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:40:16.937618 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:40:52.350209 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:40:52.350165 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:41:02.641710 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:41:02.641674 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:41:40.650440 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:41:40.650347 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:42:27.843017 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:42:27.842977 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:43:04.330970 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:04.330874 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:43:08.949771 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:08.949730 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755"] Jul 29 21:43:08.950324 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:08.950025 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755" podUID="fd926dff-0074-48fc-9843-a110ec7e1027" containerName="payload-processing" containerID="cri-o://ea9df1c7499be1ddc2e33be21319ebde3cabcfb74090a3531055e826604227c1" gracePeriod=30 Jul 29 21:43:08.955434 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:08.955393 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd"] Jul 29 21:43:08.955714 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:08.955656 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd" podUID="91c2102b-7d59-43fd-94a8-8e9dac0d6562" containerName="payload-pre-processing" containerID="cri-o://7b10a1207ac4e807166153078eb4af4afcffaa71ee33f3ffb3f8748e0e15a177" gracePeriod=30 Jul 29 21:43:09.225123 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.225089 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755" Jul 29 21:43:09.253266 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.253237 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd" Jul 29 21:43:09.363982 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.363942 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91c2102b-7d59-43fd-94a8-8e9dac0d6562-plugins-config-volume\") pod \"91c2102b-7d59-43fd-94a8-8e9dac0d6562\" (UID: \"91c2102b-7d59-43fd-94a8-8e9dac0d6562\") " Jul 29 21:43:09.364193 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.364040 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd926dff-0074-48fc-9843-a110ec7e1027-plugins-config-volume\") pod \"fd926dff-0074-48fc-9843-a110ec7e1027\" (UID: \"fd926dff-0074-48fc-9843-a110ec7e1027\") " Jul 29 21:43:09.364193 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.364109 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5w9l\" (UniqueName: \"kubernetes.io/projected/91c2102b-7d59-43fd-94a8-8e9dac0d6562-kube-api-access-c5w9l\") pod \"91c2102b-7d59-43fd-94a8-8e9dac0d6562\" (UID: \"91c2102b-7d59-43fd-94a8-8e9dac0d6562\") " Jul 29 21:43:09.364193 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.364149 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrkp5\" (UniqueName: \"kubernetes.io/projected/fd926dff-0074-48fc-9843-a110ec7e1027-kube-api-access-lrkp5\") pod \"fd926dff-0074-48fc-9843-a110ec7e1027\" (UID: \"fd926dff-0074-48fc-9843-a110ec7e1027\") " Jul 29 21:43:09.364439 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.364405 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd926dff-0074-48fc-9843-a110ec7e1027-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fd926dff-0074-48fc-9843-a110ec7e1027" (UID: "fd926dff-0074-48fc-9843-a110ec7e1027"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:43:09.364439 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.364405 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91c2102b-7d59-43fd-94a8-8e9dac0d6562-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "91c2102b-7d59-43fd-94a8-8e9dac0d6562" (UID: "91c2102b-7d59-43fd-94a8-8e9dac0d6562"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:43:09.366306 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.366278 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91c2102b-7d59-43fd-94a8-8e9dac0d6562-kube-api-access-c5w9l" (OuterVolumeSpecName: "kube-api-access-c5w9l") pod "91c2102b-7d59-43fd-94a8-8e9dac0d6562" (UID: "91c2102b-7d59-43fd-94a8-8e9dac0d6562"). InnerVolumeSpecName "kube-api-access-c5w9l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:43:09.366306 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.366296 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd926dff-0074-48fc-9843-a110ec7e1027-kube-api-access-lrkp5" (OuterVolumeSpecName: "kube-api-access-lrkp5") pod "fd926dff-0074-48fc-9843-a110ec7e1027" (UID: "fd926dff-0074-48fc-9843-a110ec7e1027"). InnerVolumeSpecName "kube-api-access-lrkp5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:43:09.465859 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.465747 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd926dff-0074-48fc-9843-a110ec7e1027-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:43:09.465859 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.465796 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5w9l\" (UniqueName: \"kubernetes.io/projected/91c2102b-7d59-43fd-94a8-8e9dac0d6562-kube-api-access-c5w9l\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:43:09.465859 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.465806 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrkp5\" (UniqueName: \"kubernetes.io/projected/fd926dff-0074-48fc-9843-a110ec7e1027-kube-api-access-lrkp5\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:43:09.465859 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.465815 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91c2102b-7d59-43fd-94a8-8e9dac0d6562-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:43:09.901058 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.901008 2559 generic.go:356] "Generic (PLEG): container finished" podID="fd926dff-0074-48fc-9843-a110ec7e1027" containerID="ea9df1c7499be1ddc2e33be21319ebde3cabcfb74090a3531055e826604227c1" exitCode=0 Jul 29 21:43:09.901276 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.901066 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755" Jul 29 21:43:09.901276 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.901080 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755" event={"ID":"fd926dff-0074-48fc-9843-a110ec7e1027","Type":"ContainerDied","Data":"ea9df1c7499be1ddc2e33be21319ebde3cabcfb74090a3531055e826604227c1"} Jul 29 21:43:09.901276 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.901135 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755" event={"ID":"fd926dff-0074-48fc-9843-a110ec7e1027","Type":"ContainerDied","Data":"92621f763d6444135fbf294d73fa8e048350b48776f903679e18686027f363d7"} Jul 29 21:43:09.901276 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.901165 2559 scope.go:117] "RemoveContainer" containerID="ea9df1c7499be1ddc2e33be21319ebde3cabcfb74090a3531055e826604227c1" Jul 29 21:43:09.902359 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.902331 2559 generic.go:356] "Generic (PLEG): container finished" podID="91c2102b-7d59-43fd-94a8-8e9dac0d6562" containerID="7b10a1207ac4e807166153078eb4af4afcffaa71ee33f3ffb3f8748e0e15a177" exitCode=0 Jul 29 21:43:09.902481 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.902377 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd" Jul 29 21:43:09.902481 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.902417 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd" event={"ID":"91c2102b-7d59-43fd-94a8-8e9dac0d6562","Type":"ContainerDied","Data":"7b10a1207ac4e807166153078eb4af4afcffaa71ee33f3ffb3f8748e0e15a177"} Jul 29 21:43:09.902481 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.902469 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd" event={"ID":"91c2102b-7d59-43fd-94a8-8e9dac0d6562","Type":"ContainerDied","Data":"244f6271b4df53f43544a37ed5c8807479d63ceeca7cab8b0b930528686796a5"} Jul 29 21:43:09.910139 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.910115 2559 scope.go:117] "RemoveContainer" containerID="ea9df1c7499be1ddc2e33be21319ebde3cabcfb74090a3531055e826604227c1" Jul 29 21:43:09.910499 ip-10-0-138-67 kubenswrapper[2559]: E0729 21:43:09.910474 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea9df1c7499be1ddc2e33be21319ebde3cabcfb74090a3531055e826604227c1\": container with ID starting with ea9df1c7499be1ddc2e33be21319ebde3cabcfb74090a3531055e826604227c1 not found: ID does not exist" containerID="ea9df1c7499be1ddc2e33be21319ebde3cabcfb74090a3531055e826604227c1" Jul 29 21:43:09.910612 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.910507 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea9df1c7499be1ddc2e33be21319ebde3cabcfb74090a3531055e826604227c1"} err="failed to get container status \"ea9df1c7499be1ddc2e33be21319ebde3cabcfb74090a3531055e826604227c1\": rpc error: code = NotFound desc = could not find container \"ea9df1c7499be1ddc2e33be21319ebde3cabcfb74090a3531055e826604227c1\": container with ID starting with ea9df1c7499be1ddc2e33be21319ebde3cabcfb74090a3531055e826604227c1 not found: ID does not exist" Jul 29 21:43:09.910612 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.910526 2559 scope.go:117] "RemoveContainer" containerID="7b10a1207ac4e807166153078eb4af4afcffaa71ee33f3ffb3f8748e0e15a177" Jul 29 21:43:09.919749 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.919730 2559 scope.go:117] "RemoveContainer" containerID="7b10a1207ac4e807166153078eb4af4afcffaa71ee33f3ffb3f8748e0e15a177" Jul 29 21:43:09.920110 ip-10-0-138-67 kubenswrapper[2559]: E0729 21:43:09.920088 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b10a1207ac4e807166153078eb4af4afcffaa71ee33f3ffb3f8748e0e15a177\": container with ID starting with 7b10a1207ac4e807166153078eb4af4afcffaa71ee33f3ffb3f8748e0e15a177 not found: ID does not exist" containerID="7b10a1207ac4e807166153078eb4af4afcffaa71ee33f3ffb3f8748e0e15a177" Jul 29 21:43:09.920191 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.920118 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b10a1207ac4e807166153078eb4af4afcffaa71ee33f3ffb3f8748e0e15a177"} err="failed to get container status \"7b10a1207ac4e807166153078eb4af4afcffaa71ee33f3ffb3f8748e0e15a177\": rpc error: code = NotFound desc = could not find container \"7b10a1207ac4e807166153078eb4af4afcffaa71ee33f3ffb3f8748e0e15a177\": container with ID starting with 7b10a1207ac4e807166153078eb4af4afcffaa71ee33f3ffb3f8748e0e15a177 not found: ID does not exist" Jul 29 21:43:09.930211 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.930177 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755"] Jul 29 21:43:09.934685 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.934651 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5f7d51-6fbc8bdcfd-6r755"] Jul 29 21:43:09.951948 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.951916 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd"] Jul 29 21:43:09.957407 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:09.957377 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-5f7d51-864bd46db4-p4ttd"] Jul 29 21:43:11.025795 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:11.025761 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91c2102b-7d59-43fd-94a8-8e9dac0d6562" path="/var/lib/kubelet/pods/91c2102b-7d59-43fd-94a8-8e9dac0d6562/volumes" Jul 29 21:43:11.026230 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:11.026151 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd926dff-0074-48fc-9843-a110ec7e1027" path="/var/lib/kubelet/pods/fd926dff-0074-48fc-9843-a110ec7e1027/volumes" Jul 29 21:43:14.549414 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:14.549362 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:43:26.985990 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:26.985954 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7"] Jul 29 21:43:26.986532 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:26.986250 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7" podUID="5c43741d-a07a-491e-909b-117effce6659" containerName="payload-processing" containerID="cri-o://dc734a767e93676cde95b85b853a22c9d75a261380b15dfabe6965e05acbed49" gracePeriod=30 Jul 29 21:43:26.998868 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:26.998834 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj"] Jul 29 21:43:26.999126 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:26.999093 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj" podUID="98ed4417-f510-4574-931c-7807ec4ac780" containerName="payload-pre-processing" containerID="cri-o://3e8487220b6d5b8d100f01ce3865b6a4652632b89c12dcb910caa65c4ff1f704" gracePeriod=30 Jul 29 21:43:27.264174 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.264150 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7" Jul 29 21:43:27.277080 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.277032 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj" Jul 29 21:43:27.426623 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.426578 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98ed4417-f510-4574-931c-7807ec4ac780-plugins-config-volume\") pod \"98ed4417-f510-4574-931c-7807ec4ac780\" (UID: \"98ed4417-f510-4574-931c-7807ec4ac780\") " Jul 29 21:43:27.426831 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.426647 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbdj7\" (UniqueName: \"kubernetes.io/projected/5c43741d-a07a-491e-909b-117effce6659-kube-api-access-mbdj7\") pod \"5c43741d-a07a-491e-909b-117effce6659\" (UID: \"5c43741d-a07a-491e-909b-117effce6659\") " Jul 29 21:43:27.426831 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.426670 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c43741d-a07a-491e-909b-117effce6659-plugins-config-volume\") pod \"5c43741d-a07a-491e-909b-117effce6659\" (UID: \"5c43741d-a07a-491e-909b-117effce6659\") " Jul 29 21:43:27.426831 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.426719 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ww9kw\" (UniqueName: \"kubernetes.io/projected/98ed4417-f510-4574-931c-7807ec4ac780-kube-api-access-ww9kw\") pod \"98ed4417-f510-4574-931c-7807ec4ac780\" (UID: \"98ed4417-f510-4574-931c-7807ec4ac780\") " Jul 29 21:43:27.427118 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.427095 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98ed4417-f510-4574-931c-7807ec4ac780-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "98ed4417-f510-4574-931c-7807ec4ac780" (UID: "98ed4417-f510-4574-931c-7807ec4ac780"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:43:27.427200 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.427112 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c43741d-a07a-491e-909b-117effce6659-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5c43741d-a07a-491e-909b-117effce6659" (UID: "5c43741d-a07a-491e-909b-117effce6659"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 21:43:27.428874 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.428844 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c43741d-a07a-491e-909b-117effce6659-kube-api-access-mbdj7" (OuterVolumeSpecName: "kube-api-access-mbdj7") pod "5c43741d-a07a-491e-909b-117effce6659" (UID: "5c43741d-a07a-491e-909b-117effce6659"). InnerVolumeSpecName "kube-api-access-mbdj7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:43:27.428992 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.428907 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98ed4417-f510-4574-931c-7807ec4ac780-kube-api-access-ww9kw" (OuterVolumeSpecName: "kube-api-access-ww9kw") pod "98ed4417-f510-4574-931c-7807ec4ac780" (UID: "98ed4417-f510-4574-931c-7807ec4ac780"). InnerVolumeSpecName "kube-api-access-ww9kw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 21:43:27.528309 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.528214 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ww9kw\" (UniqueName: \"kubernetes.io/projected/98ed4417-f510-4574-931c-7807ec4ac780-kube-api-access-ww9kw\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:43:27.528309 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.528244 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98ed4417-f510-4574-931c-7807ec4ac780-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:43:27.528309 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.528254 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbdj7\" (UniqueName: \"kubernetes.io/projected/5c43741d-a07a-491e-909b-117effce6659-kube-api-access-mbdj7\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:43:27.528309 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.528263 2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c43741d-a07a-491e-909b-117effce6659-plugins-config-volume\") on node \"ip-10-0-138-67.ec2.internal\" DevicePath \"\"" Jul 29 21:43:27.997774 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.997731 2559 generic.go:356] "Generic (PLEG): container finished" podID="5c43741d-a07a-491e-909b-117effce6659" containerID="dc734a767e93676cde95b85b853a22c9d75a261380b15dfabe6965e05acbed49" exitCode=0 Jul 29 21:43:27.998322 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.997793 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7" Jul 29 21:43:27.998322 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.997823 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7" event={"ID":"5c43741d-a07a-491e-909b-117effce6659","Type":"ContainerDied","Data":"dc734a767e93676cde95b85b853a22c9d75a261380b15dfabe6965e05acbed49"} Jul 29 21:43:27.998322 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.997933 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7" event={"ID":"5c43741d-a07a-491e-909b-117effce6659","Type":"ContainerDied","Data":"642217fce5a1c8150a8ccb48ea8ebedec264deac3ecb5ab12d64671d5dea6fe6"} Jul 29 21:43:27.998322 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.997969 2559 scope.go:117] "RemoveContainer" containerID="dc734a767e93676cde95b85b853a22c9d75a261380b15dfabe6965e05acbed49" Jul 29 21:43:27.999530 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.999414 2559 generic.go:356] "Generic (PLEG): container finished" podID="98ed4417-f510-4574-931c-7807ec4ac780" containerID="3e8487220b6d5b8d100f01ce3865b6a4652632b89c12dcb910caa65c4ff1f704" exitCode=0 Jul 29 21:43:27.999530 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.999444 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj" event={"ID":"98ed4417-f510-4574-931c-7807ec4ac780","Type":"ContainerDied","Data":"3e8487220b6d5b8d100f01ce3865b6a4652632b89c12dcb910caa65c4ff1f704"} Jul 29 21:43:27.999530 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.999481 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj" event={"ID":"98ed4417-f510-4574-931c-7807ec4ac780","Type":"ContainerDied","Data":"4f910346fbbea1af2a1a62be2c2d4d367afd6ac6e794c624c071f0d8182e0b21"} Jul 29 21:43:27.999530 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:27.999490 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj" Jul 29 21:43:28.013252 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:28.013231 2559 scope.go:117] "RemoveContainer" containerID="dc734a767e93676cde95b85b853a22c9d75a261380b15dfabe6965e05acbed49" Jul 29 21:43:28.013600 ip-10-0-138-67 kubenswrapper[2559]: E0729 21:43:28.013575 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc734a767e93676cde95b85b853a22c9d75a261380b15dfabe6965e05acbed49\": container with ID starting with dc734a767e93676cde95b85b853a22c9d75a261380b15dfabe6965e05acbed49 not found: ID does not exist" containerID="dc734a767e93676cde95b85b853a22c9d75a261380b15dfabe6965e05acbed49" Jul 29 21:43:28.013650 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:28.013610 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc734a767e93676cde95b85b853a22c9d75a261380b15dfabe6965e05acbed49"} err="failed to get container status \"dc734a767e93676cde95b85b853a22c9d75a261380b15dfabe6965e05acbed49\": rpc error: code = NotFound desc = could not find container \"dc734a767e93676cde95b85b853a22c9d75a261380b15dfabe6965e05acbed49\": container with ID starting with dc734a767e93676cde95b85b853a22c9d75a261380b15dfabe6965e05acbed49 not found: ID does not exist" Jul 29 21:43:28.013650 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:28.013631 2559 scope.go:117] "RemoveContainer" containerID="3e8487220b6d5b8d100f01ce3865b6a4652632b89c12dcb910caa65c4ff1f704" Jul 29 21:43:28.021779 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:28.021754 2559 scope.go:117] "RemoveContainer" containerID="3e8487220b6d5b8d100f01ce3865b6a4652632b89c12dcb910caa65c4ff1f704" Jul 29 21:43:28.022105 ip-10-0-138-67 kubenswrapper[2559]: E0729 21:43:28.022085 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e8487220b6d5b8d100f01ce3865b6a4652632b89c12dcb910caa65c4ff1f704\": container with ID starting with 3e8487220b6d5b8d100f01ce3865b6a4652632b89c12dcb910caa65c4ff1f704 not found: ID does not exist" containerID="3e8487220b6d5b8d100f01ce3865b6a4652632b89c12dcb910caa65c4ff1f704" Jul 29 21:43:28.022162 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:28.022116 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e8487220b6d5b8d100f01ce3865b6a4652632b89c12dcb910caa65c4ff1f704"} err="failed to get container status \"3e8487220b6d5b8d100f01ce3865b6a4652632b89c12dcb910caa65c4ff1f704\": rpc error: code = NotFound desc = could not find container \"3e8487220b6d5b8d100f01ce3865b6a4652632b89c12dcb910caa65c4ff1f704\": container with ID starting with 3e8487220b6d5b8d100f01ce3865b6a4652632b89c12dcb910caa65c4ff1f704 not found: ID does not exist" Jul 29 21:43:28.027580 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:28.027549 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7"] Jul 29 21:43:28.032088 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:28.032060 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-eee583-756d5767c8-hrlh7"] Jul 29 21:43:28.042451 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:28.042419 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj"] Jul 29 21:43:28.047458 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:28.047429 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-eee583-f6bfdd67b-q94wj"] Jul 29 21:43:29.025716 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:29.025680 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c43741d-a07a-491e-909b-117effce6659" path="/var/lib/kubelet/pods/5c43741d-a07a-491e-909b-117effce6659/volumes" Jul 29 21:43:29.026143 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:29.026044 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98ed4417-f510-4574-931c-7807ec4ac780" path="/var/lib/kubelet/pods/98ed4417-f510-4574-931c-7807ec4ac780/volumes" Jul 29 21:43:29.930930 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:29.930885 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-j98nb"] Jul 29 21:43:40.816469 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:40.816440 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74f7855d79-slmhp_2366750e-0e86-48ec-9b08-0a118c63572b/copy-manifests/0.log" Jul 29 21:43:40.823979 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:40.823950 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74f7855d79-slmhp_2366750e-0e86-48ec-9b08-0a118c63572b/manager/0.log" Jul 29 21:43:41.696470 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:41.696440 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29755995-nxqjb_5f85175c-9e15-4a93-add1-9101f1897f37/cleanup/0.log" Jul 29 21:43:44.027269 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:44.027238 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-j98nb_b63ac234-7510-4273-bf41-933e21cea421/limitador/0.log" Jul 29 21:43:44.651802 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:44.651764 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-cl4t7_0a9764a5-7986-4e6c-86e6-c90d9059bc61/discovery/0.log" Jul 29 21:43:46.099802 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:46.099768 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjkbtj_82dcc2f8-97a1-4f9c-8f69-aaf2ae947ada/main/0.log" Jul 29 21:43:54.822135 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:54.822102 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-sv4n2_b26164a7-1242-459e-8152-4a15c188d00e/global-pull-secret-syncer/0.log" Jul 29 21:43:54.982436 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:54.982399 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rbkc4_d837c888-2f72-4f39-8338-da84d1b30057/konnectivity-agent/0.log" Jul 29 21:43:55.050614 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:43:55.050583 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-67.ec2.internal_644f10f3f902f0c1cbb90a43f1a164ef/haproxy/0.log" Jul 29 21:44:00.679481 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:00.679448 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-j98nb_b63ac234-7510-4273-bf41-933e21cea421/limitador/0.log" Jul 29 21:44:02.791198 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:02.791153 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-sgttt_6a729414-a205-4b68-bee8-0c24d964a385/cluster-monitoring-operator/0.log" Jul 29 21:44:02.908696 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:02.908664 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-b46b855dd-4wd56_35a9a79c-5aeb-48f9-86b9-410263ba6c28/metrics-server/0.log" Jul 29 21:44:02.934400 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:02.934358 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-zrmw4_74f7a0e9-1364-4196-846e-d936c3fa4c7b/monitoring-plugin/0.log" Jul 29 21:44:03.063101 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:03.062994 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f5blf_5797ad79-9ff2-4480-b284-ad4ce5000915/node-exporter/0.log" Jul 29 21:44:03.090693 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:03.090656 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f5blf_5797ad79-9ff2-4480-b284-ad4ce5000915/kube-rbac-proxy/0.log" Jul 29 21:44:03.116343 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:03.116319 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f5blf_5797ad79-9ff2-4480-b284-ad4ce5000915/init-textfile/0.log" Jul 29 21:44:03.230064 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:03.230018 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-5bcs7_936bcf63-8ee2-461b-8e1e-783a3c0ac208/kube-rbac-proxy-main/0.log" Jul 29 21:44:03.263533 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:03.263495 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-5bcs7_936bcf63-8ee2-461b-8e1e-783a3c0ac208/kube-rbac-proxy-self/0.log" Jul 29 21:44:03.294647 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:03.294616 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-5bcs7_936bcf63-8ee2-461b-8e1e-783a3c0ac208/openshift-state-metrics/0.log" Jul 29 21:44:03.495916 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:03.495874 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-zvmj7_b2288f7e-999c-4428-8e45-8e05672a6e78/prometheus-operator/0.log" Jul 29 21:44:03.510670 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:03.510630 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-zvmj7_b2288f7e-999c-4428-8e45-8e05672a6e78/kube-rbac-proxy/0.log" Jul 29 21:44:03.564287 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:03.564255 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85fdbbd49c-7zqpc_85617a10-471f-47a7-8fa8-31f7ced84817/telemeter-client/0.log" Jul 29 21:44:03.583801 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:03.583763 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85fdbbd49c-7zqpc_85617a10-471f-47a7-8fa8-31f7ced84817/reload/0.log" Jul 29 21:44:03.604517 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:03.604483 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85fdbbd49c-7zqpc_85617a10-471f-47a7-8fa8-31f7ced84817/kube-rbac-proxy/0.log" Jul 29 21:44:05.035385 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.035342 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj"] Jul 29 21:44:05.035814 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.035702 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98ed4417-f510-4574-931c-7807ec4ac780" containerName="payload-pre-processing" Jul 29 21:44:05.035814 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.035715 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="98ed4417-f510-4574-931c-7807ec4ac780" containerName="payload-pre-processing" Jul 29 21:44:05.035814 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.035733 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c43741d-a07a-491e-909b-117effce6659" containerName="payload-processing" Jul 29 21:44:05.035814 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.035739 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c43741d-a07a-491e-909b-117effce6659" containerName="payload-processing" Jul 29 21:44:05.035814 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.035754 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91c2102b-7d59-43fd-94a8-8e9dac0d6562" containerName="payload-pre-processing" Jul 29 21:44:05.035814 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.035761 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="91c2102b-7d59-43fd-94a8-8e9dac0d6562" containerName="payload-pre-processing" Jul 29 21:44:05.035814 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.035771 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd926dff-0074-48fc-9843-a110ec7e1027" containerName="payload-processing" Jul 29 21:44:05.035814 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.035777 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd926dff-0074-48fc-9843-a110ec7e1027" containerName="payload-processing" Jul 29 21:44:05.036108 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.035826 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="91c2102b-7d59-43fd-94a8-8e9dac0d6562" containerName="payload-pre-processing" Jul 29 21:44:05.036108 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.035834 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c43741d-a07a-491e-909b-117effce6659" containerName="payload-processing" Jul 29 21:44:05.036108 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.035840 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd926dff-0074-48fc-9843-a110ec7e1027" containerName="payload-processing" Jul 29 21:44:05.036108 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.035848 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="98ed4417-f510-4574-931c-7807ec4ac780" containerName="payload-pre-processing" Jul 29 21:44:05.038998 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.038979 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" Jul 29 21:44:05.041949 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.041918 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pwwzw\"/\"default-dockercfg-5976b\"" Jul 29 21:44:05.042134 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.041950 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pwwzw\"/\"openshift-service-ca.crt\"" Jul 29 21:44:05.042738 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.042718 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pwwzw\"/\"kube-root-ca.crt\"" Jul 29 21:44:05.050735 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.050708 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj"] Jul 29 21:44:05.168523 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.168475 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbppf\" (UniqueName: \"kubernetes.io/projected/519c0612-8178-47c1-bf30-1e58aa73052f-kube-api-access-dbppf\") pod \"perf-node-gather-daemonset-j5vcj\" (UID: \"519c0612-8178-47c1-bf30-1e58aa73052f\") " pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" Jul 29 21:44:05.168716 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.168535 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/519c0612-8178-47c1-bf30-1e58aa73052f-proc\") pod \"perf-node-gather-daemonset-j5vcj\" (UID: \"519c0612-8178-47c1-bf30-1e58aa73052f\") " pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" Jul 29 21:44:05.168716 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.168591 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/519c0612-8178-47c1-bf30-1e58aa73052f-sys\") pod \"perf-node-gather-daemonset-j5vcj\" (UID: \"519c0612-8178-47c1-bf30-1e58aa73052f\") " pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" Jul 29 21:44:05.168716 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.168647 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/519c0612-8178-47c1-bf30-1e58aa73052f-podres\") pod \"perf-node-gather-daemonset-j5vcj\" (UID: \"519c0612-8178-47c1-bf30-1e58aa73052f\") " pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" Jul 29 21:44:05.168716 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.168696 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/519c0612-8178-47c1-bf30-1e58aa73052f-lib-modules\") pod \"perf-node-gather-daemonset-j5vcj\" (UID: \"519c0612-8178-47c1-bf30-1e58aa73052f\") " pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" Jul 29 21:44:05.269677 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.269640 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/519c0612-8178-47c1-bf30-1e58aa73052f-proc\") pod \"perf-node-gather-daemonset-j5vcj\" (UID: \"519c0612-8178-47c1-bf30-1e58aa73052f\") " pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" Jul 29 21:44:05.269677 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.269676 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/519c0612-8178-47c1-bf30-1e58aa73052f-sys\") pod \"perf-node-gather-daemonset-j5vcj\" (UID: \"519c0612-8178-47c1-bf30-1e58aa73052f\") " pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" Jul 29 21:44:05.269896 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.269700 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/519c0612-8178-47c1-bf30-1e58aa73052f-podres\") pod \"perf-node-gather-daemonset-j5vcj\" (UID: \"519c0612-8178-47c1-bf30-1e58aa73052f\") " pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" Jul 29 21:44:05.269896 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.269742 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/519c0612-8178-47c1-bf30-1e58aa73052f-lib-modules\") pod \"perf-node-gather-daemonset-j5vcj\" (UID: \"519c0612-8178-47c1-bf30-1e58aa73052f\") " pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" Jul 29 21:44:05.269896 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.269772 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/519c0612-8178-47c1-bf30-1e58aa73052f-proc\") pod \"perf-node-gather-daemonset-j5vcj\" (UID: \"519c0612-8178-47c1-bf30-1e58aa73052f\") " pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" Jul 29 21:44:05.269896 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.269786 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbppf\" (UniqueName: \"kubernetes.io/projected/519c0612-8178-47c1-bf30-1e58aa73052f-kube-api-access-dbppf\") pod \"perf-node-gather-daemonset-j5vcj\" (UID: \"519c0612-8178-47c1-bf30-1e58aa73052f\") " pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" Jul 29 21:44:05.269896 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.269772 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/519c0612-8178-47c1-bf30-1e58aa73052f-sys\") pod \"perf-node-gather-daemonset-j5vcj\" (UID: \"519c0612-8178-47c1-bf30-1e58aa73052f\") " pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" Jul 29 21:44:05.269896 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.269864 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/519c0612-8178-47c1-bf30-1e58aa73052f-podres\") pod \"perf-node-gather-daemonset-j5vcj\" (UID: \"519c0612-8178-47c1-bf30-1e58aa73052f\") " pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" Jul 29 21:44:05.269896 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.269889 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/519c0612-8178-47c1-bf30-1e58aa73052f-lib-modules\") pod \"perf-node-gather-daemonset-j5vcj\" (UID: \"519c0612-8178-47c1-bf30-1e58aa73052f\") " pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" Jul 29 21:44:05.279822 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.279786 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbppf\" (UniqueName: \"kubernetes.io/projected/519c0612-8178-47c1-bf30-1e58aa73052f-kube-api-access-dbppf\") pod \"perf-node-gather-daemonset-j5vcj\" (UID: \"519c0612-8178-47c1-bf30-1e58aa73052f\") " pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" Jul 29 21:44:05.349167 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.349033 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" Jul 29 21:44:05.490661 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:05.490632 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj"] Jul 29 21:44:05.494534 ip-10-0-138-67 kubenswrapper[2559]: W0729 21:44:05.494500 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod519c0612_8178_47c1_bf30_1e58aa73052f.slice/crio-1f56baba097a7878e968c639fab74f20c501ca1a5e785c3c95b677a959ed4c7d WatchSource:0}: Error finding container 1f56baba097a7878e968c639fab74f20c501ca1a5e785c3c95b677a959ed4c7d: Status 404 returned error can't find the container with id 1f56baba097a7878e968c639fab74f20c501ca1a5e785c3c95b677a959ed4c7d Jul 29 21:44:06.185031 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:06.184990 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" event={"ID":"519c0612-8178-47c1-bf30-1e58aa73052f","Type":"ContainerStarted","Data":"eda419323f1168e273bbb0be58a60fd04a00bc1f787632914420a866fb26cdb6"} Jul 29 21:44:06.185462 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:06.185042 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" event={"ID":"519c0612-8178-47c1-bf30-1e58aa73052f","Type":"ContainerStarted","Data":"1f56baba097a7878e968c639fab74f20c501ca1a5e785c3c95b677a959ed4c7d"} Jul 29 21:44:06.185462 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:06.185124 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" Jul 29 21:44:06.200503 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:06.200447 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" podStartSLOduration=1.200428971 podStartE2EDuration="1.200428971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:44:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:44:06.200221433 +0000 UTC m=+4741.715527668" watchObservedRunningTime="2026-07-29 21:44:06.200428971 +0000 UTC m=+4741.715735195" Jul 29 21:44:06.273496 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:06.273452 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-lmtmp_00b36d47-033f-4ba9-a2a0-8c9e56ba4393/download-server/0.log" Jul 29 21:44:08.011434 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:08.011404 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nlftr_a072f2ce-aee2-412d-90cf-9ecc83a109df/dns/0.log" Jul 29 21:44:08.031189 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:08.031159 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nlftr_a072f2ce-aee2-412d-90cf-9ecc83a109df/kube-rbac-proxy/0.log" Jul 29 21:44:08.053701 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:08.053673 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-d7c5v_42288536-6377-4883-90bb-69df7fc92db3/dns-node-resolver/0.log" Jul 29 21:44:08.663527 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:08.663496 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-647c86dff8-ls4m7_ceddaa08-0819-4908-bf58-03b8798c1a84/registry/0.log" Jul 29 21:44:08.739346 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:08.739315 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zmsn6_c5ec56df-1bb7-42be-846b-17648ead013f/node-ca/0.log" Jul 29 21:44:09.920558 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:09.920511 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-cl4t7_0a9764a5-7986-4e6c-86e6-c90d9059bc61/discovery/0.log" Jul 29 21:44:11.019650 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:11.019612 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qcswx_655beca7-e0da-4b4c-9001-322b63d9f11a/serve-healthcheck-canary/0.log" Jul 29 21:44:11.479648 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:11.479541 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-s7t8m_fc9b1faf-5638-4fd3-ae81-d579fbf2f56f/insights-operator/0.log" Jul 29 21:44:11.482699 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:11.482665 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-s7t8m_fc9b1faf-5638-4fd3-ae81-d579fbf2f56f/insights-operator/1.log" Jul 29 21:44:11.504024 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:11.503989 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dw8kz_3e7cc3ba-05fb-4d6a-9211-21196d16d214/kube-rbac-proxy/0.log" Jul 29 21:44:11.524715 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:11.524676 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dw8kz_3e7cc3ba-05fb-4d6a-9211-21196d16d214/exporter/0.log" Jul 29 21:44:11.543466 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:11.543435 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dw8kz_3e7cc3ba-05fb-4d6a-9211-21196d16d214/extractor/0.log" Jul 29 21:44:12.199786 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:12.199758 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pwwzw/perf-node-gather-daemonset-j5vcj" Jul 29 21:44:14.607973 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:14.607939 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74f7855d79-slmhp_2366750e-0e86-48ec-9b08-0a118c63572b/manager/0.log" Jul 29 21:44:14.636217 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:14.636188 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74f7855d79-slmhp_2366750e-0e86-48ec-9b08-0a118c63572b/copy-manifests/0.log" Jul 29 21:44:15.502468 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:15.502432 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-cl4t7_0a9764a5-7986-4e6c-86e6-c90d9059bc61/discovery/0.log" Jul 29 21:44:16.337367 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:16.337335 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5c7786b84c-v5lc8_02057b30-6881-43c6-82cf-96d87bf24444/manager/0.log" Jul 29 21:44:16.361485 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:16.361449 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-fl7mc_2123db89-19f4-4b1a-b680-bf53295d25e7/openshift-lws-operator/0.log" Jul 29 21:44:17.456786 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:17.456746 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-cl4t7_0a9764a5-7986-4e6c-86e6-c90d9059bc61/discovery/0.log" Jul 29 21:44:22.990383 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:22.990355 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nv7t7_42d531a1-1dba-44b6-a335-7c32dc39dc4f/kube-multus-additional-cni-plugins/0.log" Jul 29 21:44:23.012602 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:23.012573 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nv7t7_42d531a1-1dba-44b6-a335-7c32dc39dc4f/egress-router-binary-copy/0.log" Jul 29 21:44:23.033938 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:23.033904 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nv7t7_42d531a1-1dba-44b6-a335-7c32dc39dc4f/cni-plugins/0.log" Jul 29 21:44:23.060466 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:23.060437 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nv7t7_42d531a1-1dba-44b6-a335-7c32dc39dc4f/bond-cni-plugin/0.log" Jul 29 21:44:23.082726 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:23.082692 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nv7t7_42d531a1-1dba-44b6-a335-7c32dc39dc4f/routeoverride-cni/0.log" Jul 29 21:44:23.115733 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:23.115702 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nv7t7_42d531a1-1dba-44b6-a335-7c32dc39dc4f/whereabouts-cni-bincopy/0.log" Jul 29 21:44:23.133656 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:23.133625 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nv7t7_42d531a1-1dba-44b6-a335-7c32dc39dc4f/whereabouts-cni/0.log" Jul 29 21:44:23.228638 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:23.228601 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-phbfq_278e281a-76db-41fb-8cef-3cdbd6a94701/kube-multus/0.log" Jul 29 21:44:23.314553 ip-10-0-138-67 kubenswrapper[2559]: I0729 21:44:23.314516 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vsmj9_e8f8cbb6-aaf2-4e92-a7fe-351fadad61a1/network-metrics-daemon/0.log"