Jul 10 18:17:21.496365 ip-10-0-143-54 systemd[1]: Starting Kubernetes Kubelet... Jul 10 18:17:23.467651 ip-10-0-143-54 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 18:17:23.467651 ip-10-0-143-54 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 18:17:23.467651 ip-10-0-143-54 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 18:17:23.467651 ip-10-0-143-54 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 18:17:23.467651 ip-10-0-143-54 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 18:17:23.467651 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.385297 2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 18:17:23.467651 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389565 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:17:23.467651 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389586 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:17:23.467651 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389590 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:17:23.467651 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389595 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:17:23.467651 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389599 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:17:23.467651 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389602 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:17:23.467651 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389605 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:17:23.467651 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389608 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:17:23.468633 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389627 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:17:23.468633 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389630 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:17:23.468633 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389632 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:17:23.468633 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389635 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:17:23.468633 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389638 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:17:23.468633 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389641 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:17:23.468633 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389644 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:17:23.468633 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389647 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:17:23.468633 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389650 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:17:23.468633 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389652 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:17:23.468633 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389658 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:17:23.468633 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389661 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:17:23.468633 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389664 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:17:23.468633 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389666 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:17:23.468633 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389669 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:17:23.468633 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389672 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:17:23.468633 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389675 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:17:23.468633 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389678 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:17:23.468633 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389680 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:17:23.468993 ip-10-0-143-54 systemd[1]: Started Kubernetes Kubelet. Jul 10 18:17:23.470547 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389683 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:17:23.470547 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389685 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:17:23.470547 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389688 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:17:23.470547 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389691 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:17:23.470547 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389693 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:17:23.470547 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389696 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:17:23.470547 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389699 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:17:23.470547 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389701 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:17:23.470547 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389704 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:17:23.470547 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389707 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:17:23.470547 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389709 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:17:23.470547 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389715 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:17:23.470547 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389719 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:17:23.470547 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389723 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:17:23.470547 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389726 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:17:23.470547 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389729 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:17:23.470547 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389731 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:17:23.470547 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389734 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:17:23.470547 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389737 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:17:23.471518 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389739 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:17:23.471518 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389742 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:17:23.471518 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389745 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:17:23.471518 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389748 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:17:23.471518 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389751 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:17:23.471518 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389754 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:17:23.471518 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389757 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:17:23.471518 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389760 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:17:23.471518 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389762 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:17:23.471518 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389765 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:17:23.471518 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389767 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:17:23.471518 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389770 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:17:23.471518 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389773 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:17:23.471518 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389776 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:17:23.471518 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389780 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:17:23.471518 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389782 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:17:23.471518 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389786 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:17:23.471518 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389788 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:17:23.471518 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389791 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:17:23.471518 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389794 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:17:23.472366 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389797 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:17:23.472366 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389799 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:17:23.472366 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389802 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:17:23.472366 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389804 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:17:23.472366 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389807 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:17:23.472366 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389809 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:17:23.472366 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389812 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:17:23.472366 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389814 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:17:23.472366 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389817 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:17:23.472366 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389819 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:17:23.472366 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389822 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:17:23.472366 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389825 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:17:23.472366 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389828 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:17:23.472366 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389833 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:17:23.472366 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389836 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:17:23.472366 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389840 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:17:23.472366 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389842 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:17:23.472366 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389845 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:17:23.472366 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389848 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:17:23.472366 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389852 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:17:23.472366 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389855 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:17:23.473462 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389857 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:17:23.473462 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389860 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:17:23.473462 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389863 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:17:23.473462 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389865 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:17:23.473462 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389867 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:17:23.473462 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389871 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:17:23.473462 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389875 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:17:23.473462 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389878 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:17:23.473462 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389880 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:17:23.473462 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389883 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:17:23.473462 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389886 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:17:23.473462 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.389888 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:17:23.473462 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390782 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:17:23.473462 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390792 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:17:23.473462 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390796 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:17:23.473462 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390800 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:17:23.473462 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390804 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:17:23.473462 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390808 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:17:23.473462 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390812 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:17:23.474446 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390816 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:17:23.474446 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390823 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:17:23.474446 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390829 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:17:23.474446 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390834 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:17:23.474446 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390839 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:17:23.474446 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390843 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:17:23.474446 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390847 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:17:23.474446 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390851 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:17:23.474446 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390855 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:17:23.474446 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390859 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:17:23.474446 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390863 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:17:23.474446 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390867 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:17:23.474446 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390870 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:17:23.474446 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390874 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:17:23.474446 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390878 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:17:23.474446 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390882 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:17:23.474446 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390886 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:17:23.474446 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390891 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:17:23.474446 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390895 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:17:23.475364 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390899 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:17:23.475364 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390903 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:17:23.475364 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390908 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:17:23.475364 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390911 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:17:23.475364 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390915 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:17:23.475364 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390919 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:17:23.475364 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390923 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:17:23.475364 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390927 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:17:23.475364 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390930 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:17:23.475364 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390934 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:17:23.475364 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390937 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:17:23.475364 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390941 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:17:23.475364 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390946 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:17:23.475364 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390950 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:17:23.475364 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390953 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:17:23.475364 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390957 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:17:23.475364 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390963 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:17:23.475364 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390967 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:17:23.475364 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390971 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:17:23.475364 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390975 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:17:23.721463 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390979 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:17:23.721463 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390982 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:17:23.721463 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390987 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:17:23.721463 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390991 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:17:23.721463 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390995 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:17:23.721463 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.390998 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:17:23.721463 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391006 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:17:23.721463 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391010 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:17:23.721463 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391014 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:17:23.721463 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391018 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:17:23.721463 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391022 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:17:23.721463 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391027 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:17:23.721463 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391031 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:17:23.721463 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391035 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:17:23.721463 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391038 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:17:23.721463 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391043 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:17:23.721463 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391046 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:17:23.721463 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391050 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:17:23.721463 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391056 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:17:23.723828 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391061 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:17:23.723828 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391065 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:17:23.723828 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391070 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:17:23.723828 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391074 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:17:23.723828 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391078 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:17:23.723828 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391081 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:17:23.723828 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391085 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:17:23.723828 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391090 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:17:23.723828 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391094 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:17:23.723828 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391097 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:17:23.723828 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391101 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:17:23.723828 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391105 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:17:23.723828 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391110 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:17:23.723828 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391116 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:17:23.723828 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391120 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:17:23.723828 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391124 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:17:23.723828 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391128 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:17:23.723828 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391133 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:17:23.723828 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391136 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:17:23.723828 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391141 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:17:23.724609 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391145 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:17:23.724609 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391149 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:17:23.724609 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391153 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:17:23.724609 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391160 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:17:23.724609 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391164 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:17:23.724609 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391167 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:17:23.724609 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391171 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:17:23.724609 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391174 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:17:23.724609 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391178 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:17:23.724609 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391182 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:17:23.724609 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391186 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:17:23.724609 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391190 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:17:23.724609 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391193 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:17:23.724609 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391198 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:17:23.724609 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391858 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:17:23.724609 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391867 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:17:23.724609 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391871 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:17:23.724609 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391877 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:17:23.724609 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391882 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:17:23.725392 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391886 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:17:23.725392 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391890 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:17:23.725392 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391894 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:17:23.725392 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391898 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:17:23.725392 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391902 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:17:23.725392 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391906 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:17:23.725392 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391910 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:17:23.725392 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391914 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:17:23.725392 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391918 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:17:23.725392 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391929 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:17:23.725392 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391933 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:17:23.725392 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391937 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:17:23.725392 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391943 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:17:23.725392 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391947 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:17:23.725392 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391951 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:17:23.725392 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391955 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:17:23.725392 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391961 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:17:23.725392 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391965 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:17:23.725392 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391969 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:17:23.725392 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391972 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:17:23.726018 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391976 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:17:23.726018 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391980 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:17:23.726018 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391983 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:17:23.726018 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391987 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:17:23.726018 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391991 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:17:23.726018 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391995 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:17:23.726018 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.391999 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:17:23.726018 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392003 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:17:23.726018 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392006 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:17:23.726018 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392010 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:17:23.726018 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392014 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:17:23.726018 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392020 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:17:23.726018 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392025 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:17:23.726018 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392030 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:17:23.726018 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392034 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:17:23.726018 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392038 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:17:23.726018 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392042 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:17:23.726018 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392047 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:17:23.726018 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392051 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:17:23.726018 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392055 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:17:23.726662 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392058 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:17:23.726662 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392062 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:17:23.726662 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392078 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:17:23.726662 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392083 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:17:23.726662 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392089 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:17:23.726662 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392093 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:17:23.726662 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392097 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:17:23.726662 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392101 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:17:23.726662 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392108 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:17:23.726662 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392111 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:17:23.726662 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392115 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:17:23.726662 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392118 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:17:23.726662 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392122 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:17:23.726662 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392126 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:17:23.726662 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392129 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:17:23.726662 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392133 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:17:23.726662 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392137 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:17:23.726662 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392140 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:17:23.726662 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392144 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:17:23.727274 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392148 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:17:23.727274 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392152 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:17:23.727274 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392156 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:17:23.727274 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392159 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:17:23.727274 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392164 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:17:23.727274 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392168 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:17:23.727274 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392172 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:17:23.727274 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392175 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:17:23.727274 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392179 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:17:23.727274 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392183 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:17:23.727274 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392187 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:17:23.727274 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392191 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:17:23.727274 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392195 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:17:23.727274 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392199 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:17:23.727274 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392203 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:17:23.727274 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392207 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:17:23.727274 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392221 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:17:23.727274 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392227 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:17:23.727274 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392231 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:17:23.727274 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392235 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:17:23.727914 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392238 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:17:23.727914 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392245 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:17:23.727914 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392249 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:17:23.727914 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392253 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:17:23.727914 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392257 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:17:23.727914 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392261 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:17:23.727914 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392265 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:17:23.727914 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392269 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:17:23.727914 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392273 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:17:23.727914 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392276 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:17:23.727914 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392280 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:17:23.727914 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392284 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:17:23.727914 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392288 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:17:23.727914 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392291 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:17:23.727914 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.392296 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:17:23.727914 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392396 2571 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 18:17:23.727914 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392419 2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 18:17:23.727914 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392431 2571 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 18:17:23.727914 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392438 2571 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 18:17:23.727914 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392445 2571 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 18:17:23.727914 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392450 2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392456 2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392462 2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392467 2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392472 2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392477 2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392482 2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392486 2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392491 2571 flags.go:64] FLAG: --cgroup-root="" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392504 2571 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392509 2571 flags.go:64] FLAG: --client-ca-file="" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392513 2571 flags.go:64] FLAG: --cloud-provider="external" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392518 2571 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392528 2571 flags.go:64] FLAG: --cluster-domain="" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392532 2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392537 2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392541 2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392546 2571 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392552 2571 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392556 2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392561 2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392566 2571 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392571 2571 flags.go:64] FLAG: --contention-profiling="false" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392576 2571 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 18:17:23.728461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392580 2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392584 2571 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392588 2571 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392595 2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392599 2571 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392604 2571 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392608 2571 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392626 2571 flags.go:64] FLAG: --enable-server="true" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392631 2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392642 2571 flags.go:64] FLAG: --event-burst="100" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392646 2571 flags.go:64] FLAG: --event-qps="50" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392650 2571 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392654 2571 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392659 2571 flags.go:64] FLAG: --eviction-hard="" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392665 2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392669 2571 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392673 2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392680 2571 flags.go:64] FLAG: --eviction-soft="" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392691 2571 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392695 2571 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392700 2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392706 2571 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392710 2571 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392715 2571 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392719 2571 flags.go:64] FLAG: --feature-gates="" Jul 10 18:17:23.729127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392726 2571 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392730 2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392734 2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392739 2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392743 2571 flags.go:64] FLAG: --healthz-port="10248" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392747 2571 flags.go:64] FLAG: --help="false" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392751 2571 flags.go:64] FLAG: --hostname-override="ip-10-0-143-54.ec2.internal" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392756 2571 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392760 2571 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392764 2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392769 2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392774 2571 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392778 2571 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392782 2571 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392787 2571 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392791 2571 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392796 2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392800 2571 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392804 2571 flags.go:64] FLAG: --kube-reserved="" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392809 2571 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392813 2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392817 2571 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392821 2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392826 2571 flags.go:64] FLAG: --lock-file="" Jul 10 18:17:23.729856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392831 2571 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392836 2571 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392850 2571 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392858 2571 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392864 2571 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392868 2571 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392872 2571 flags.go:64] FLAG: --logging-format="text" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392877 2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392882 2571 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392886 2571 flags.go:64] FLAG: --manifest-url="" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392890 2571 flags.go:64] FLAG: --manifest-url-header="" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392897 2571 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392901 2571 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392907 2571 flags.go:64] FLAG: --max-pods="110" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392912 2571 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392916 2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392920 2571 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392925 2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392929 2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392934 2571 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392938 2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392949 2571 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392953 2571 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392958 2571 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 18:17:23.730518 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392962 2571 flags.go:64] FLAG: --pod-cidr="" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392966 2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392974 2571 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392978 2571 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392982 2571 flags.go:64] FLAG: --pods-per-core="0" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392986 2571 flags.go:64] FLAG: --port="10250" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392990 2571 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.392994 2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-059850d77c916501c" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393001 2571 flags.go:64] FLAG: --qos-reserved="" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393005 2571 flags.go:64] FLAG: --read-only-port="10255" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393010 2571 flags.go:64] FLAG: --register-node="true" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393021 2571 flags.go:64] FLAG: --register-with-taints="" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393027 2571 flags.go:64] FLAG: --registry-burst="10" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393033 2571 flags.go:64] FLAG: --registry-qps="5" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393037 2571 flags.go:64] FLAG: --reserved-cpus="" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393041 2571 flags.go:64] FLAG: --reserved-memory="" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393047 2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393051 2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393056 2571 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393060 2571 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393064 2571 flags.go:64] FLAG: --runonce="false" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393068 2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393072 2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393077 2571 flags.go:64] FLAG: --seccomp-default="false" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393081 2571 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 18:17:23.731160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393085 2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393089 2571 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393093 2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393098 2571 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393102 2571 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393106 2571 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393111 2571 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393115 2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393119 2571 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393123 2571 flags.go:64] FLAG: --system-cgroups="" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393127 2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393135 2571 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393139 2571 flags.go:64] FLAG: --tls-cert-file="" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393144 2571 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393152 2571 flags.go:64] FLAG: --tls-min-version="" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393159 2571 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393163 2571 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393167 2571 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393171 2571 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393186 2571 flags.go:64] FLAG: --v="2" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393194 2571 flags.go:64] FLAG: --version="false" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393200 2571 flags.go:64] FLAG: --vmodule="" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393206 2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.393212 2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393441 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:17:23.731872 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393447 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:17:23.732896 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393451 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:17:23.732896 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393455 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:17:23.732896 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393459 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:17:23.732896 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393463 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:17:23.732896 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393467 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:17:23.732896 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393471 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:17:23.732896 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393474 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:17:23.732896 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393478 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:17:23.732896 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393482 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:17:23.732896 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393486 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:17:23.732896 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393489 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:17:23.732896 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393493 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:17:23.732896 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393497 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:17:23.732896 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393500 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:17:23.732896 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393529 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:17:23.732896 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393534 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:17:23.732896 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393538 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:17:23.732896 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393543 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:17:23.732896 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393547 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:17:23.732896 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393551 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:17:23.733466 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393555 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:17:23.733466 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393561 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:17:23.733466 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393565 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:17:23.733466 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393569 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:17:23.733466 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393573 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:17:23.733466 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393577 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:17:23.733466 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393591 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:17:23.733466 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393596 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:17:23.733466 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393601 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:17:23.733466 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393606 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:17:23.733466 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393627 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:17:23.733466 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393632 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:17:23.733466 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393635 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:17:23.733466 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393640 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:17:23.733466 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393644 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:17:23.733466 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393648 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:17:23.733466 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393652 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:17:23.733466 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393656 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:17:23.733466 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393660 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:17:23.733466 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393663 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:17:23.734059 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393666 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:17:23.734059 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393670 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:17:23.734059 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393673 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:17:23.734059 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393677 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:17:23.734059 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393681 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:17:23.734059 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393684 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:17:23.734059 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393688 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:17:23.734059 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393692 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:17:23.734059 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393696 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:17:23.734059 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393700 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:17:23.734059 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393703 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:17:23.734059 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393707 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:17:23.734059 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393710 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:17:23.734059 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393716 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:17:23.734059 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393720 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:17:23.734059 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393723 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:17:23.734059 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393727 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:17:23.734059 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393731 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:17:23.734059 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393737 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:17:23.734059 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393748 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:17:23.734837 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393752 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:17:23.734837 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393756 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:17:23.734837 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393760 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:17:23.734837 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393772 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:17:23.734837 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393775 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:17:23.734837 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393779 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:17:23.734837 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393784 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:17:23.734837 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393788 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:17:23.734837 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393792 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:17:23.734837 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393796 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:17:23.734837 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393800 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:17:23.734837 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393803 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:17:23.734837 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393807 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:17:23.734837 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393811 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:17:23.734837 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393814 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:17:23.734837 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393819 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:17:23.734837 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393823 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:17:23.734837 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393826 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:17:23.734837 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393830 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:17:23.735459 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393833 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:17:23.735459 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393837 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:17:23.735459 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393843 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:17:23.735459 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393847 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:17:23.735459 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393851 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:17:23.735459 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393855 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:17:23.735459 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393861 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:17:23.735459 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393865 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:17:23.735459 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393869 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:17:23.735459 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393873 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:17:23.735459 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393879 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:17:23.735459 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393885 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:17:23.735459 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393890 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:17:23.735459 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393902 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:17:23.735459 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393906 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:17:23.735459 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393910 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:17:23.735459 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393914 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:17:23.735459 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.393918 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:17:23.736272 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.394421 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 18:17:23.736272 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.403005 2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 18:17:23.736272 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.403027 2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 18:17:23.736272 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.403842 2571 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 18:17:23.736272 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.403850 2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 18:17:23.736272 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403914 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:17:23.736272 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403921 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:17:23.736272 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403925 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:17:23.736272 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403928 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:17:23.736272 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403931 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:17:23.736272 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403935 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:17:23.736272 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403937 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:17:23.736272 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403940 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:17:23.736272 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403943 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:17:23.736272 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403945 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:17:23.736905 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403948 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:17:23.736905 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403951 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:17:23.736905 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403953 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:17:23.736905 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403956 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:17:23.736905 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403959 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:17:23.736905 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403963 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:17:23.736905 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403966 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:17:23.736905 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403968 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:17:23.736905 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403971 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:17:23.736905 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403974 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:17:23.736905 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403977 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:17:23.736905 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403980 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:17:23.736905 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403982 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:17:23.736905 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403985 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:17:23.736905 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403988 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:17:23.736905 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403991 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:17:23.736905 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403993 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:17:23.736905 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403996 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:17:23.736905 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.403998 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:17:23.736905 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404001 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:17:23.736905 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404004 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:17:24.009115 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404006 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:17:24.009115 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404009 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:17:24.009115 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404012 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:17:24.009115 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404015 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:17:24.009115 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404017 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:17:24.009115 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404020 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:17:24.009115 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404023 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:17:24.009115 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404025 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:17:24.009115 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404032 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:17:24.009115 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404036 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:17:24.009115 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404040 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:17:24.009115 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404044 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:17:24.009115 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404047 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:17:24.009115 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404050 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:17:24.009115 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404053 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:17:24.009115 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404056 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:17:24.009115 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404060 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:17:24.009115 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404063 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:17:24.009115 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404066 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:17:24.009780 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404068 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:17:24.009780 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404071 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:17:24.009780 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404074 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:17:24.009780 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404076 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:17:24.009780 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404079 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:17:24.009780 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404081 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:17:24.009780 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404084 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:17:24.009780 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404086 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:17:24.009780 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404089 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:17:24.009780 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404091 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:17:24.009780 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404094 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:17:24.009780 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404096 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:17:24.009780 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404099 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:17:24.009780 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404102 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:17:24.009780 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404104 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:17:24.009780 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404107 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:17:24.009780 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404109 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:17:24.009780 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404112 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:17:24.009780 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404114 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:17:24.010454 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404117 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:17:24.010454 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404119 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:17:24.010454 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404123 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:17:24.010454 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404126 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:17:24.010454 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404128 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:17:24.010454 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404131 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:17:24.010454 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404133 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:17:24.010454 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404137 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:17:24.010454 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404141 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:17:24.010454 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404144 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:17:24.010454 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404148 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:17:24.010454 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404151 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:17:24.010454 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404156 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:17:24.010454 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404159 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:17:24.010454 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404162 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:17:24.010454 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404165 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:17:24.010454 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404167 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:17:24.010454 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404170 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:17:24.010454 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404172 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:17:24.011022 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404175 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:17:24.011022 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404177 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:17:24.011022 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404180 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:17:24.011022 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404182 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:17:24.011022 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404185 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:17:24.011022 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404188 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:17:24.011022 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404190 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:17:24.011022 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404193 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:17:24.011022 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404196 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:17:24.011022 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404199 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:17:24.011022 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404204 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:17:24.011022 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.404209 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 18:17:24.011022 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404329 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 18:17:24.011022 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404335 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 18:17:24.011022 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404338 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 18:17:24.011502 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404341 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 18:17:24.011502 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404346 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 18:17:24.011502 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404348 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 18:17:24.011502 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404351 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 18:17:24.011502 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404354 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 18:17:24.011502 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404357 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 18:17:24.011502 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404359 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 18:17:24.011502 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404362 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 18:17:24.011502 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404364 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 18:17:24.011502 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404368 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 18:17:24.011502 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404370 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 18:17:24.011502 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404373 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 18:17:24.011502 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404376 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 18:17:24.011502 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404378 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 18:17:24.011502 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404381 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 18:17:24.011502 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404383 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 18:17:24.011502 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404386 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 18:17:24.011502 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404388 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 18:17:24.011502 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404391 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 18:17:24.011502 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404393 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 18:17:24.012568 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404396 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 18:17:24.012568 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404399 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 18:17:24.012568 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404401 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 18:17:24.012568 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404404 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 18:17:24.012568 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404406 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 18:17:24.012568 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404409 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 18:17:24.012568 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404412 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 18:17:24.012568 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404416 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 18:17:24.012568 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404418 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 18:17:24.012568 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404421 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 18:17:24.012568 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404426 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 18:17:24.012568 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404429 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 18:17:24.012568 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404431 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 18:17:24.012568 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404436 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 18:17:24.012568 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404439 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 18:17:24.012568 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404441 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 18:17:24.012568 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404444 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 18:17:24.012568 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404447 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 18:17:24.012568 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404449 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 18:17:24.012568 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404451 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 18:17:24.013176 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404454 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 18:17:24.013176 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404456 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 18:17:24.013176 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404460 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 18:17:24.013176 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404462 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 18:17:24.013176 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404466 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 18:17:24.013176 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404469 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 18:17:24.013176 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404472 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 18:17:24.013176 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404475 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 18:17:24.013176 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404478 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 18:17:24.013176 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404481 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 18:17:24.013176 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404483 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 18:17:24.013176 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404486 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 18:17:24.013176 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404489 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 18:17:24.013176 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404493 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 18:17:24.013176 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404496 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 18:17:24.013176 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404499 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 18:17:24.013176 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404502 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 18:17:24.013176 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404504 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 18:17:24.013176 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404507 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 18:17:24.013742 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404511 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 18:17:24.013742 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404514 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 18:17:24.013742 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404516 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 18:17:24.013742 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404519 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 18:17:24.013742 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404521 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 18:17:24.013742 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404524 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 18:17:24.013742 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404528 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 18:17:24.013742 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404530 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 18:17:24.013742 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404533 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 18:17:24.013742 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404536 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 18:17:24.013742 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404539 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 18:17:24.013742 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404541 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 18:17:24.013742 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404544 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 18:17:24.013742 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404546 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 18:17:24.013742 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404549 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 18:17:24.013742 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404551 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 18:17:24.013742 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404555 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 18:17:24.013742 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404558 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 18:17:24.013742 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404560 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 18:17:24.013742 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404563 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 18:17:24.014427 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404566 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 18:17:24.014427 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404568 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 18:17:24.014427 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404571 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 18:17:24.014427 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404573 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 18:17:24.014427 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404576 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 18:17:24.014427 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404578 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 18:17:24.014427 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404581 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 18:17:24.014427 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404583 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 18:17:24.014427 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404586 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 18:17:24.014427 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404588 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 18:17:24.014427 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404591 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 18:17:24.014427 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404593 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 18:17:24.014427 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404598 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 18:17:24.014427 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404600 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 18:17:24.014427 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404603 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 18:17:24.014427 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404606 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 18:17:24.014427 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:23.404608 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 18:17:24.014934 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.404628 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 18:17:24.014934 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.404766 2571 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 18:17:24.014934 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.406873 2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 18:17:24.014934 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.408085 2571 server.go:1019] "Starting client certificate rotation" Jul 10 18:17:24.014934 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.408192 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 18:17:24.014934 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.408233 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 18:17:24.014934 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.410540 2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 18:17:24.014934 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.411930 2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 18:17:24.014934 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.423520 2571 log.go:25] "Validated CRI v1 runtime API" Jul 10 18:17:24.014934 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.428002 2571 log.go:25] "Validated CRI v1 image API" Jul 10 18:17:24.014934 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.428958 2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 18:17:24.014934 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.433501 2571 fs.go:135] Filesystem UUIDs: map[1c1f4634-727a-457c-ab26-4075a5d7ff52:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 8c461160-ae78-48b6-a38a-aaf758c36aaa:/dev/nvme0n1p4] Jul 10 18:17:24.015313 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.433517 2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 18:17:24.015564 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.440957 2571 manager.go:216] Machine: {Timestamp:2026-07-10 18:17:23.439504207 +0000 UTC m=+0.338773306 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102740 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ab690909841dbe5012d6e22855a68 SystemUUID:ec2ab690-9098-41db-e501-2d6e22855a68 BootID:2d2bdd7f-b633-4d2f-be08-96be2bc54a7a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c6:dc:9c:7e:4b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c6:dc:9c:7e:4b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:9a:cb:c3:43:2b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 18:17:24.015564 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.441072 2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 18:17:24.015564 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.441170 2571 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 18:17:24.015564 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.444077 2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 18:17:24.015564 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.444113 2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-54.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 18:17:24.015564 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.444780 2571 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 18:17:24.015564 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.444791 2571 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 18:17:24.015564 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.444799 2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.445499 2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.446041 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.446305 2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.451148 2571 state_mem.go:36] "Initialized new in-memory state store" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.451291 2571 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.451324 2571 kubelet.go:486] "Attempting to sync node with API server" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.451334 2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.451351 2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.451361 2571 kubelet.go:398] "Adding apiserver pod source" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.451370 2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.451885 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.451906 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.454135 2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.456018 2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.456057 2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.457124 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.457140 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.457147 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.457153 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.457161 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.457168 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.457175 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.457180 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.457188 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.457193 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.457202 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.457207 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.457990 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.458000 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.461509 2571 server.go:1268] "Started kubelet" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.461639 2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-54.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.461627 2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:23.461683 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.461689 2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:23.461736 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-54.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.461756 2571 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.463940 2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 18:17:24.015892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.464295 2571 server.go:310] "Adding debug handlers to kubelet server" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:23.466878 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-54.ec2.internal.18c1007826bdd658 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-54.ec2.internal,UID:ip-10-0-143-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-54.ec2.internal,},FirstTimestamp:2026-07-10 18:17:23.461477976 +0000 UTC m=+0.360747114,LastTimestamp:2026-07-10 18:17:23.461477976 +0000 UTC m=+0.360747114,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-54.ec2.internal,}" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.468900 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bnxh2" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:23.469177 2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.470095 2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.470116 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:23.470296 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-54.ec2.internal\" not found" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.470678 2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.470686 2571 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.470703 2571 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.470784 2571 reconstruct.go:97] "Volume reconstruction finished" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.470793 2571 reconciler.go:29] "Reconciler: start to sync state" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.471143 2571 factory.go:55] Registering systemd factory Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.471160 2571 factory.go:223] Registration of the systemd container factory successfully Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.471441 2571 factory.go:153] Registering CRI-O factory Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.471456 2571 factory.go:223] Registration of the crio container factory successfully Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.471515 2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.471539 2571 factory.go:103] Registering Raw factory Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.471556 2571 manager.go:1196] Started watching for new ooms in manager Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.471914 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bnxh2" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.472228 2571 manager.go:349] Starting recovery of all containers Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:23.472502 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:23.472830 2571 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-54.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.483310 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.487865 2571 manager.go:354] Recovery completed Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.492374 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.494264 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.494303 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.494315 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.496635 2571 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.496646 2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.496663 2571 state_mem.go:36] "Initialized new in-memory state store" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.496991 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.497734 2571 policy_none.go:49] "None policy: Start" Jul 10 18:17:24.236089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.497758 2571 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.497769 2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.497924 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.497950 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.497959 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.498682 2571 policy_none.go:47] "Start" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.538266 2571 manager.go:337] "Starting Device Plugin manager" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:23.538309 2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.538322 2571 server.go:87] "Starting device plugin registration server" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.538595 2571 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.538608 2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.538707 2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.538786 2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.538792 2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:23.539411 2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:23.539458 2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-54.ec2.internal\" not found" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.638798 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.640663 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.640698 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.640712 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.640747 2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-54.ec2.internal" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.647058 2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-54.ec2.internal" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:23.647084 2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-54.ec2.internal\": node \"ip-10-0-143-54.ec2.internal\" not found" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:23.660578 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-54.ec2.internal\" not found" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.676973 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.677012 2571 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.677037 2571 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:23.677067 2571 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.679125 2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:23.760946 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-54.ec2.internal\" not found" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.777353 2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-54.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-54.ec2.internal"] Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.777448 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.779996 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.780027 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.780037 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.781427 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.781576 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-54.ec2.internal" Jul 10 18:17:24.237749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.781607 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:17:24.238856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.782239 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:17:24.238856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.782264 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:17:24.238856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.782274 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:17:24.238856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.782240 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:17:24.238856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.782342 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:17:24.238856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.782354 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:17:24.238856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.783474 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-54.ec2.internal" Jul 10 18:17:24.238856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.783496 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 18:17:24.238856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.784196 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeHasSufficientMemory" Jul 10 18:17:24.238856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.784224 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 18:17:24.238856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.784240 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeHasSufficientPID" Jul 10 18:17:24.238856 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:23.820484 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-54.ec2.internal\" not found" node="ip-10-0-143-54.ec2.internal" Jul 10 18:17:24.238856 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:23.824815 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-54.ec2.internal\" not found" node="ip-10-0-143-54.ec2.internal" Jul 10 18:17:24.238856 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:23.861141 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-54.ec2.internal\" not found" Jul 10 18:17:24.238856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.873362 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b69d880dba39cd41f6208dd45b48e550-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-54.ec2.internal\" (UID: \"b69d880dba39cd41f6208dd45b48e550\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-54.ec2.internal" Jul 10 18:17:24.238856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.873395 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b69d880dba39cd41f6208dd45b48e550-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-54.ec2.internal\" (UID: \"b69d880dba39cd41f6208dd45b48e550\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-54.ec2.internal" Jul 10 18:17:24.238856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.873416 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33a81504b2c888f51c5efff3d00523ac-config\") pod \"kube-apiserver-proxy-ip-10-0-143-54.ec2.internal\" (UID: \"33a81504b2c888f51c5efff3d00523ac\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-54.ec2.internal" Jul 10 18:17:24.238856 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:23.961969 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-54.ec2.internal\" not found" Jul 10 18:17:24.238856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.974290 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b69d880dba39cd41f6208dd45b48e550-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-54.ec2.internal\" (UID: \"b69d880dba39cd41f6208dd45b48e550\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-54.ec2.internal" Jul 10 18:17:24.238856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.974320 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b69d880dba39cd41f6208dd45b48e550-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-54.ec2.internal\" (UID: \"b69d880dba39cd41f6208dd45b48e550\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-54.ec2.internal" Jul 10 18:17:24.238856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.974338 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33a81504b2c888f51c5efff3d00523ac-config\") pod \"kube-apiserver-proxy-ip-10-0-143-54.ec2.internal\" (UID: \"33a81504b2c888f51c5efff3d00523ac\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-54.ec2.internal" Jul 10 18:17:24.239507 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.974390 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b69d880dba39cd41f6208dd45b48e550-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-54.ec2.internal\" (UID: \"b69d880dba39cd41f6208dd45b48e550\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-54.ec2.internal" Jul 10 18:17:24.239507 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.974422 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/33a81504b2c888f51c5efff3d00523ac-config\") pod \"kube-apiserver-proxy-ip-10-0-143-54.ec2.internal\" (UID: \"33a81504b2c888f51c5efff3d00523ac\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-54.ec2.internal" Jul 10 18:17:24.239507 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:23.974472 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b69d880dba39cd41f6208dd45b48e550-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-54.ec2.internal\" (UID: \"b69d880dba39cd41f6208dd45b48e550\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-54.ec2.internal" Jul 10 18:17:24.239507 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:24.062449 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-54.ec2.internal\" not found" Jul 10 18:17:24.239507 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:24.121962 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-54.ec2.internal" Jul 10 18:17:24.239507 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:24.127802 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-54.ec2.internal" Jul 10 18:17:24.239507 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:24.163238 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-54.ec2.internal\" not found" Jul 10 18:17:24.502960 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:24.264233 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-54.ec2.internal\" not found" Jul 10 18:17:24.502960 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:24.364810 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-54.ec2.internal\" not found" Jul 10 18:17:24.502960 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:24.376515 2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:17:24.502960 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:24.407240 2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 18:17:24.502960 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:24.407427 2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 18:17:24.502960 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:24.407427 2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 18:17:24.502960 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:24.465443 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-54.ec2.internal\" not found" Jul 10 18:17:24.502960 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:24.477725 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 18:12:23 +0000 UTC" deadline="2028-03-28 00:34:46.795919476 +0000 UTC" Jul 10 18:17:24.502960 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:24.477752 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15030h17m22.318170498s" Jul 10 18:17:24.755266 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:24.566022 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-54.ec2.internal\" not found" Jul 10 18:17:24.755266 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:24.666499 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-54.ec2.internal\" not found" Jul 10 18:17:25.025473 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:24.766923 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-54.ec2.internal\" not found" Jul 10 18:17:25.025473 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:24.775323 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33a81504b2c888f51c5efff3d00523ac.slice/crio-53cf6f36442bb17200def9d7e966c3ef7f495b947e0f476f8b6ca893f7ec3326 WatchSource:0}: Error finding container 53cf6f36442bb17200def9d7e966c3ef7f495b947e0f476f8b6ca893f7ec3326: Status 404 returned error can't find the container with id 53cf6f36442bb17200def9d7e966c3ef7f495b947e0f476f8b6ca893f7ec3326 Jul 10 18:17:25.025473 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:24.775625 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb69d880dba39cd41f6208dd45b48e550.slice/crio-5ed0af85fff0c59bc0b321d21825d32243e8d6effcd0e57c446f3ab7454cd62c WatchSource:0}: Error finding container 5ed0af85fff0c59bc0b321d21825d32243e8d6effcd0e57c446f3ab7454cd62c: Status 404 returned error can't find the container with id 5ed0af85fff0c59bc0b321d21825d32243e8d6effcd0e57c446f3ab7454cd62c Jul 10 18:17:25.025473 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:24.782680 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:17:25.025473 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:24.867932 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-54.ec2.internal\" not found" Jul 10 18:17:25.025473 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:24.968745 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-54.ec2.internal\" not found" Jul 10 18:17:25.027146 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.027019 2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:17:25.279486 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.065866 2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:17:25.279486 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.070298 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-54.ec2.internal" Jul 10 18:17:25.279486 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.083604 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 18:17:25.279486 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.084722 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-54.ec2.internal" Jul 10 18:17:25.279486 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.093959 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.452569 2571 apiserver.go:52] "Watching apiserver" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.460684 2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.461065 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-ndsqx","openshift-ovn-kubernetes/ovnkube-node-7r9vz","kube-system/global-pull-secret-syncer-9rqql","kube-system/kube-apiserver-proxy-ip-10-0-143-54.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8","openshift-image-registry/node-ca-6jpkg","openshift-multus/multus-additional-cni-plugins-fzc82","openshift-network-diagnostics/network-check-target-lftkj","openshift-network-operator/iptables-alerter-9d2qb","kube-system/konnectivity-agent-hn6g2","openshift-cluster-node-tuning-operator/tuned-9gx7f","openshift-dns/node-resolver-kt69c","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-54.ec2.internal","openshift-multus/multus-zdlvx"] Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.462811 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hn6g2" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.464216 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.465205 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.465322 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lw2wh\"" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.465547 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:25.465609 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9rqql" podUID="91c02392-14bc-403d-ab9f-131ed9213d28" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.466256 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.466743 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.466953 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.467183 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.467386 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-z7qt9\"" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.468170 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.469249 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.469274 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.469305 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.469392 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.469500 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-t95gg\"" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.469752 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6jpkg" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.470881 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.470994 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9d2qb" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.472748 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:25.472819 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lftkj" podUID="0a36554d-7c38-4893-9dca-ceac16436f2e" Jul 10 18:17:25.536674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.472879 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.473083 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ps8n2\"" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.473497 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.473545 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.473501 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.473735 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.474134 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.474264 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cp2qn\"" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.474353 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-v2zrk\"" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.474500 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.474940 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.475142 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.476107 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.477373 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:25.477459 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsqx" podUID="29ea8cc9-50a4-4990-a85c-b8c47aa81f0b" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.478889 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.479088 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.480010 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-k2b72\"" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.480302 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.480463 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.480489 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.480736 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.481756 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.482444 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-host-run-netns\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.483390 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kt69c" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.482480 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-etc-sysctl-conf\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.538504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484178 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-var-lib-kubelet\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.540416 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484194 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0bc69439-7cb2-4cda-8a15-a1904bc12785-tmp\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.540416 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484215 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/91c02392-14bc-403d-ab9f-131ed9213d28-kubelet-config\") pod \"global-pull-secret-syncer-9rqql\" (UID: \"91c02392-14bc-403d-ab9f-131ed9213d28\") " pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:25.540416 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484240 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-cnibin\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.540416 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484265 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-host-var-lib-cni-multus\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.540416 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484289 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-etc-sysconfig\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.540416 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484311 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-etc-systemd\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.540416 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484333 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-host\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.540416 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484355 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 18:17:25.540416 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484356 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6pgk\" (UniqueName: \"kubernetes.io/projected/0bc69439-7cb2-4cda-8a15-a1904bc12785-kube-api-access-z6pgk\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.540416 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484445 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/628f9bd4-aa72-4a8d-a8ad-31ddd10e208c-etc-selinux\") pod \"aws-ebs-csi-driver-node-z8kh8\" (UID: \"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.540416 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484471 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a3929aa3-2abd-489c-b7fd-c80e982db8e8-host\") pod \"node-ca-6jpkg\" (UID: \"a3929aa3-2abd-489c-b7fd-c80e982db8e8\") " pod="openshift-image-registry/node-ca-6jpkg" Jul 10 18:17:25.540416 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484494 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-host-run-k8s-cni-cncf-io\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.540416 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484515 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-lib-modules\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.540416 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484535 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/628f9bd4-aa72-4a8d-a8ad-31ddd10e208c-socket-dir\") pod \"aws-ebs-csi-driver-node-z8kh8\" (UID: \"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.541258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484555 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/628f9bd4-aa72-4a8d-a8ad-31ddd10e208c-device-dir\") pod \"aws-ebs-csi-driver-node-z8kh8\" (UID: \"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.541258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484575 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/517ac3e6-81be-4bbf-adb4-0cd53dfbf79a-host-slash\") pod \"iptables-alerter-9d2qb\" (UID: \"517ac3e6-81be-4bbf-adb4-0cd53dfbf79a\") " pod="openshift-network-operator/iptables-alerter-9d2qb" Jul 10 18:17:25.541258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484596 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-multus-cni-dir\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.541258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484650 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-multus-conf-dir\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.541258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484683 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/91c02392-14bc-403d-ab9f-131ed9213d28-dbus\") pod \"global-pull-secret-syncer-9rqql\" (UID: \"91c02392-14bc-403d-ab9f-131ed9213d28\") " pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:25.541258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484709 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/628f9bd4-aa72-4a8d-a8ad-31ddd10e208c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z8kh8\" (UID: \"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.541258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484732 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26c2w\" (UniqueName: \"kubernetes.io/projected/a3929aa3-2abd-489c-b7fd-c80e982db8e8-kube-api-access-26c2w\") pod \"node-ca-6jpkg\" (UID: \"a3929aa3-2abd-489c-b7fd-c80e982db8e8\") " pod="openshift-image-registry/node-ca-6jpkg" Jul 10 18:17:25.541258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484755 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3fbdae78-1e8a-4b53-aff0-0f807312bd7e-agent-certs\") pod \"konnectivity-agent-hn6g2\" (UID: \"3fbdae78-1e8a-4b53-aff0-0f807312bd7e\") " pod="kube-system/konnectivity-agent-hn6g2" Jul 10 18:17:25.541258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484778 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 18:17:25.541258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484778 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-etc-modprobe-d\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.541258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484853 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-etc-kubernetes\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.541258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484856 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g5z8r\"" Jul 10 18:17:25.541258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484877 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/628f9bd4-aa72-4a8d-a8ad-31ddd10e208c-registration-dir\") pod \"aws-ebs-csi-driver-node-z8kh8\" (UID: \"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.541258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484907 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2vb9\" (UniqueName: \"kubernetes.io/projected/628f9bd4-aa72-4a8d-a8ad-31ddd10e208c-kube-api-access-z2vb9\") pod \"aws-ebs-csi-driver-node-z8kh8\" (UID: \"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.541258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484927 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-system-cni-dir\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.611936 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484950 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-cni-binary-copy\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.611936 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484972 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-etc-kubernetes\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.611936 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.484995 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckckc\" (UniqueName: \"kubernetes.io/projected/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-kube-api-access-ckckc\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.611936 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.485019 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/517ac3e6-81be-4bbf-adb4-0cd53dfbf79a-iptables-alerter-script\") pod \"iptables-alerter-9d2qb\" (UID: \"517ac3e6-81be-4bbf-adb4-0cd53dfbf79a\") " pod="openshift-network-operator/iptables-alerter-9d2qb" Jul 10 18:17:25.611936 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.485055 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-multus-socket-dir-parent\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.611936 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.485077 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-host-var-lib-cni-bin\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.611936 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.485113 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-host-var-lib-kubelet\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.611936 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.485148 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3fbdae78-1e8a-4b53-aff0-0f807312bd7e-konnectivity-ca\") pod \"konnectivity-agent-hn6g2\" (UID: \"3fbdae78-1e8a-4b53-aff0-0f807312bd7e\") " pod="kube-system/konnectivity-agent-hn6g2" Jul 10 18:17:25.611936 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.485174 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-etc-sysctl-d\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.611936 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.485207 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-run\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.611936 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.485233 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a3929aa3-2abd-489c-b7fd-c80e982db8e8-serviceca\") pod \"node-ca-6jpkg\" (UID: \"a3929aa3-2abd-489c-b7fd-c80e982db8e8\") " pod="openshift-image-registry/node-ca-6jpkg" Jul 10 18:17:25.611936 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.485262 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z27zs\" (UniqueName: \"kubernetes.io/projected/517ac3e6-81be-4bbf-adb4-0cd53dfbf79a-kube-api-access-z27zs\") pod \"iptables-alerter-9d2qb\" (UID: \"517ac3e6-81be-4bbf-adb4-0cd53dfbf79a\") " pod="openshift-network-operator/iptables-alerter-9d2qb" Jul 10 18:17:25.611936 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.485310 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-hostroot\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.611936 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.485361 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-multus-daemon-config\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.612870 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.485385 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-host-run-multus-certs\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.612870 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.485406 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-sys\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.612870 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.485445 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0bc69439-7cb2-4cda-8a15-a1904bc12785-etc-tuned\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.612870 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.485497 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret\") pod \"global-pull-secret-syncer-9rqql\" (UID: \"91c02392-14bc-403d-ab9f-131ed9213d28\") " pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:25.612870 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.485556 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/628f9bd4-aa72-4a8d-a8ad-31ddd10e208c-sys-fs\") pod \"aws-ebs-csi-driver-node-z8kh8\" (UID: \"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.612870 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.485591 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-os-release\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.612870 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.485737 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 18:17:25.612870 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.485770 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-49tkz\"" Jul 10 18:17:25.612870 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.485923 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 18:17:25.612870 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.485947 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 18:17:25.612870 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.505581 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jd7rj" Jul 10 18:17:25.612870 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.510911 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jd7rj" Jul 10 18:17:25.612870 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.571919 2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 18:17:25.612870 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.585858 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-multus-socket-dir-parent\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.612870 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.585890 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3fbdae78-1e8a-4b53-aff0-0f807312bd7e-konnectivity-ca\") pod \"konnectivity-agent-hn6g2\" (UID: \"3fbdae78-1e8a-4b53-aff0-0f807312bd7e\") " pod="kube-system/konnectivity-agent-hn6g2" Jul 10 18:17:25.612870 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.585912 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-etc-sysctl-d\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.612870 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.585966 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-run\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.612870 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586020 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrmqx\" (UniqueName: \"kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx\") pod \"network-check-target-lftkj\" (UID: \"0a36554d-7c38-4893-9dca-ceac16436f2e\") " pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:25.613856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586053 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-host-run-ovn-kubernetes\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.613856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586082 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-multus-socket-dir-parent\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.613856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586087 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs\") pod \"network-metrics-daemon-ndsqx\" (UID: \"29ea8cc9-50a4-4990-a85c-b8c47aa81f0b\") " pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:25.613856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586124 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a3929aa3-2abd-489c-b7fd-c80e982db8e8-serviceca\") pod \"node-ca-6jpkg\" (UID: \"a3929aa3-2abd-489c-b7fd-c80e982db8e8\") " pod="openshift-image-registry/node-ca-6jpkg" Jul 10 18:17:25.613856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586194 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z27zs\" (UniqueName: \"kubernetes.io/projected/517ac3e6-81be-4bbf-adb4-0cd53dfbf79a-kube-api-access-z27zs\") pod \"iptables-alerter-9d2qb\" (UID: \"517ac3e6-81be-4bbf-adb4-0cd53dfbf79a\") " pod="openshift-network-operator/iptables-alerter-9d2qb" Jul 10 18:17:25.613856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586207 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-etc-sysctl-d\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.613856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586227 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-sys\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.613856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586253 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0bc69439-7cb2-4cda-8a15-a1904bc12785-etc-tuned\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.613856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586338 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-host-run-netns\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.613856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586380 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-node-log\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.613856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586416 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret\") pod \"global-pull-secret-syncer-9rqql\" (UID: \"91c02392-14bc-403d-ab9f-131ed9213d28\") " pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:25.613856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586440 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-os-release\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.613856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586458 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-host-run-netns\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.613856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586475 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-var-lib-kubelet\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.614891 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586535 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-run\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.614891 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586558 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-sys\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.614891 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586570 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0bc69439-7cb2-4cda-8a15-a1904bc12785-tmp\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.614891 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586594 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6pgk\" (UniqueName: \"kubernetes.io/projected/0bc69439-7cb2-4cda-8a15-a1904bc12785-kube-api-access-z6pgk\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.614891 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586637 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a3929aa3-2abd-489c-b7fd-c80e982db8e8-serviceca\") pod \"node-ca-6jpkg\" (UID: \"a3929aa3-2abd-489c-b7fd-c80e982db8e8\") " pod="openshift-image-registry/node-ca-6jpkg" Jul 10 18:17:25.614891 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586638 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05e2da1a-b49d-47e5-85e8-2ce1d30d3aae-tmp-dir\") pod \"node-resolver-kt69c\" (UID: \"05e2da1a-b49d-47e5-85e8-2ce1d30d3aae\") " pod="openshift-dns/node-resolver-kt69c" Jul 10 18:17:25.614891 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:25.586665 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:25.614891 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586687 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-os-release\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.614891 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586701 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-os-release\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.614891 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586736 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-var-lib-kubelet\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.614891 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:25.586757 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret podName:91c02392-14bc-403d-ab9f-131ed9213d28 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:26.086716101 +0000 UTC m=+2.985985230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret") pod "global-pull-secret-syncer-9rqql" (UID: "91c02392-14bc-403d-ab9f-131ed9213d28") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:25.614891 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586767 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-host-run-netns\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.614891 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586777 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-cnibin\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.614891 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586775 2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 18:17:25.614891 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586812 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-host-var-lib-cni-multus\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.615790 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586856 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-etc-sysconfig\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.615790 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586881 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-etc-systemd\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.615790 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586906 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-host-kubelet\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.615790 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586943 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/628f9bd4-aa72-4a8d-a8ad-31ddd10e208c-etc-selinux\") pod \"aws-ebs-csi-driver-node-z8kh8\" (UID: \"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.615790 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586948 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-host-var-lib-cni-multus\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.615790 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586977 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a3929aa3-2abd-489c-b7fd-c80e982db8e8-host\") pod \"node-ca-6jpkg\" (UID: \"a3929aa3-2abd-489c-b7fd-c80e982db8e8\") " pod="openshift-image-registry/node-ca-6jpkg" Jul 10 18:17:25.615790 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.586992 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-cnibin\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.615790 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587031 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-etc-systemd\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.615790 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587100 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a3929aa3-2abd-489c-b7fd-c80e982db8e8-host\") pod \"node-ca-6jpkg\" (UID: \"a3929aa3-2abd-489c-b7fd-c80e982db8e8\") " pod="openshift-image-registry/node-ca-6jpkg" Jul 10 18:17:25.615790 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587104 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/628f9bd4-aa72-4a8d-a8ad-31ddd10e208c-etc-selinux\") pod \"aws-ebs-csi-driver-node-z8kh8\" (UID: \"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.615790 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587102 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-etc-sysconfig\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.615790 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587132 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/05e2da1a-b49d-47e5-85e8-2ce1d30d3aae-hosts-file\") pod \"node-resolver-kt69c\" (UID: \"05e2da1a-b49d-47e5-85e8-2ce1d30d3aae\") " pod="openshift-dns/node-resolver-kt69c" Jul 10 18:17:25.615790 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587174 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/628f9bd4-aa72-4a8d-a8ad-31ddd10e208c-socket-dir\") pod \"aws-ebs-csi-driver-node-z8kh8\" (UID: \"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.615790 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587139 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3fbdae78-1e8a-4b53-aff0-0f807312bd7e-konnectivity-ca\") pod \"konnectivity-agent-hn6g2\" (UID: \"3fbdae78-1e8a-4b53-aff0-0f807312bd7e\") " pod="kube-system/konnectivity-agent-hn6g2" Jul 10 18:17:25.615790 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587212 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/517ac3e6-81be-4bbf-adb4-0cd53dfbf79a-host-slash\") pod \"iptables-alerter-9d2qb\" (UID: \"517ac3e6-81be-4bbf-adb4-0cd53dfbf79a\") " pod="openshift-network-operator/iptables-alerter-9d2qb" Jul 10 18:17:25.616801 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587238 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-multus-cni-dir\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.616801 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587257 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/517ac3e6-81be-4bbf-adb4-0cd53dfbf79a-host-slash\") pod \"iptables-alerter-9d2qb\" (UID: \"517ac3e6-81be-4bbf-adb4-0cd53dfbf79a\") " pod="openshift-network-operator/iptables-alerter-9d2qb" Jul 10 18:17:25.616801 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587260 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-multus-conf-dir\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.616801 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587295 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-multus-conf-dir\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.616801 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587300 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pt97\" (UniqueName: \"kubernetes.io/projected/05e2da1a-b49d-47e5-85e8-2ce1d30d3aae-kube-api-access-2pt97\") pod \"node-resolver-kt69c\" (UID: \"05e2da1a-b49d-47e5-85e8-2ce1d30d3aae\") " pod="openshift-dns/node-resolver-kt69c" Jul 10 18:17:25.616801 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587320 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-multus-cni-dir\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.616801 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587330 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/628f9bd4-aa72-4a8d-a8ad-31ddd10e208c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z8kh8\" (UID: \"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.616801 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587319 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/628f9bd4-aa72-4a8d-a8ad-31ddd10e208c-socket-dir\") pod \"aws-ebs-csi-driver-node-z8kh8\" (UID: \"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.616801 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587351 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26c2w\" (UniqueName: \"kubernetes.io/projected/a3929aa3-2abd-489c-b7fd-c80e982db8e8-kube-api-access-26c2w\") pod \"node-ca-6jpkg\" (UID: \"a3929aa3-2abd-489c-b7fd-c80e982db8e8\") " pod="openshift-image-registry/node-ca-6jpkg" Jul 10 18:17:25.616801 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587371 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3fbdae78-1e8a-4b53-aff0-0f807312bd7e-agent-certs\") pod \"konnectivity-agent-hn6g2\" (UID: \"3fbdae78-1e8a-4b53-aff0-0f807312bd7e\") " pod="kube-system/konnectivity-agent-hn6g2" Jul 10 18:17:25.616801 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587390 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-etc-modprobe-d\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.616801 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587422 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/628f9bd4-aa72-4a8d-a8ad-31ddd10e208c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z8kh8\" (UID: \"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.616801 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587470 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-etc-modprobe-d\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.616801 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587523 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-etc-kubernetes\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.617579 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587553 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-var-lib-openvswitch\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.617579 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587572 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-etc-kubernetes\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.617579 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587577 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-system-cni-dir\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.617579 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587602 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-cni-binary-copy\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.617579 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587629 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-system-cni-dir\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.617579 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587637 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-etc-kubernetes\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.617579 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587658 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckckc\" (UniqueName: \"kubernetes.io/projected/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-kube-api-access-ckckc\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.617579 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587680 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-run-ovn\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.617579 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587700 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-log-socket\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.617579 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587721 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.617579 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587739 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-etc-kubernetes\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.617579 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587749 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdgg7\" (UniqueName: \"kubernetes.io/projected/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-kube-api-access-rdgg7\") pod \"network-metrics-daemon-ndsqx\" (UID: \"29ea8cc9-50a4-4990-a85c-b8c47aa81f0b\") " pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:25.617579 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587782 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/517ac3e6-81be-4bbf-adb4-0cd53dfbf79a-iptables-alerter-script\") pod \"iptables-alerter-9d2qb\" (UID: \"517ac3e6-81be-4bbf-adb4-0cd53dfbf79a\") " pod="openshift-network-operator/iptables-alerter-9d2qb" Jul 10 18:17:25.617579 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587817 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-host-var-lib-cni-bin\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.618251 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587841 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-host-var-lib-kubelet\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.618251 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587864 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-host-cni-bin\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.618251 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587897 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-host-var-lib-kubelet\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.618251 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587926 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0cafad48-3998-465d-b4c8-b5d94cac6b3c-ovnkube-config\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.618251 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587931 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-host-var-lib-cni-bin\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.618251 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587950 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-cnibin\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.618251 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.587978 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-hostroot\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.618251 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588001 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-multus-daemon-config\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.618251 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588023 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-host-run-multus-certs\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.618251 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588044 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-run-openvswitch\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.618251 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588065 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0cafad48-3998-465d-b4c8-b5d94cac6b3c-env-overrides\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.618251 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588091 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0cafad48-3998-465d-b4c8-b5d94cac6b3c-ovnkube-script-lib\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.618251 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588114 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.618251 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588114 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-cni-binary-copy\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.619016 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588137 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.619016 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588170 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-host-run-multus-certs\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.619016 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588175 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/628f9bd4-aa72-4a8d-a8ad-31ddd10e208c-sys-fs\") pod \"aws-ebs-csi-driver-node-z8kh8\" (UID: \"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.619016 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588206 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-etc-sysctl-conf\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.619016 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588225 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/628f9bd4-aa72-4a8d-a8ad-31ddd10e208c-sys-fs\") pod \"aws-ebs-csi-driver-node-z8kh8\" (UID: \"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.619016 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588258 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-hostroot\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.619016 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588260 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/517ac3e6-81be-4bbf-adb4-0cd53dfbf79a-iptables-alerter-script\") pod \"iptables-alerter-9d2qb\" (UID: \"517ac3e6-81be-4bbf-adb4-0cd53dfbf79a\") " pod="openshift-network-operator/iptables-alerter-9d2qb" Jul 10 18:17:25.619016 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588282 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-host-cni-netd\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.619016 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588304 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0cafad48-3998-465d-b4c8-b5d94cac6b3c-ovn-node-metrics-cert\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.619016 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588328 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/91c02392-14bc-403d-ab9f-131ed9213d28-kubelet-config\") pod \"global-pull-secret-syncer-9rqql\" (UID: \"91c02392-14bc-403d-ab9f-131ed9213d28\") " pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:25.619016 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588349 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-host\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.619016 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588367 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/91c02392-14bc-403d-ab9f-131ed9213d28-kubelet-config\") pod \"global-pull-secret-syncer-9rqql\" (UID: \"91c02392-14bc-403d-ab9f-131ed9213d28\") " pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:25.619016 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588367 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-etc-sysctl-conf\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.619016 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588369 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-etc-openvswitch\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.620311 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588409 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-host\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.620311 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588413 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-multus-daemon-config\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.620311 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588409 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-cni-binary-copy\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.620311 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588458 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.620311 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588482 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-host-run-k8s-cni-cncf-io\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.620311 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588510 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-lib-modules\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.620311 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588555 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-host-run-k8s-cni-cncf-io\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.620311 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588579 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-run-systemd\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.620311 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588602 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/628f9bd4-aa72-4a8d-a8ad-31ddd10e208c-device-dir\") pod \"aws-ebs-csi-driver-node-z8kh8\" (UID: \"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.620311 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588636 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-systemd-units\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.620311 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588659 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6j4m\" (UniqueName: \"kubernetes.io/projected/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-kube-api-access-j6j4m\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.620311 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588684 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0bc69439-7cb2-4cda-8a15-a1904bc12785-lib-modules\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.620311 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588686 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/91c02392-14bc-403d-ab9f-131ed9213d28-dbus\") pod \"global-pull-secret-syncer-9rqql\" (UID: \"91c02392-14bc-403d-ab9f-131ed9213d28\") " pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:25.620311 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588697 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/628f9bd4-aa72-4a8d-a8ad-31ddd10e208c-device-dir\") pod \"aws-ebs-csi-driver-node-z8kh8\" (UID: \"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.621028 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588725 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvdsv\" (UniqueName: \"kubernetes.io/projected/0cafad48-3998-465d-b4c8-b5d94cac6b3c-kube-api-access-pvdsv\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.621028 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588752 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-system-cni-dir\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.621028 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588765 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/91c02392-14bc-403d-ab9f-131ed9213d28-dbus\") pod \"global-pull-secret-syncer-9rqql\" (UID: \"91c02392-14bc-403d-ab9f-131ed9213d28\") " pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:25.621028 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588785 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/628f9bd4-aa72-4a8d-a8ad-31ddd10e208c-registration-dir\") pod \"aws-ebs-csi-driver-node-z8kh8\" (UID: \"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.621028 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588808 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2vb9\" (UniqueName: \"kubernetes.io/projected/628f9bd4-aa72-4a8d-a8ad-31ddd10e208c-kube-api-access-z2vb9\") pod \"aws-ebs-csi-driver-node-z8kh8\" (UID: \"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.621028 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588829 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/628f9bd4-aa72-4a8d-a8ad-31ddd10e208c-registration-dir\") pod \"aws-ebs-csi-driver-node-z8kh8\" (UID: \"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.621028 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.588831 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-host-slash\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.621028 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.591092 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0bc69439-7cb2-4cda-8a15-a1904bc12785-etc-tuned\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.621028 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.591323 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3fbdae78-1e8a-4b53-aff0-0f807312bd7e-agent-certs\") pod \"konnectivity-agent-hn6g2\" (UID: \"3fbdae78-1e8a-4b53-aff0-0f807312bd7e\") " pod="kube-system/konnectivity-agent-hn6g2" Jul 10 18:17:25.621028 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.594772 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0bc69439-7cb2-4cda-8a15-a1904bc12785-tmp\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.622712 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.622678 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2vb9\" (UniqueName: \"kubernetes.io/projected/628f9bd4-aa72-4a8d-a8ad-31ddd10e208c-kube-api-access-z2vb9\") pod \"aws-ebs-csi-driver-node-z8kh8\" (UID: \"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.622819 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.622766 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26c2w\" (UniqueName: \"kubernetes.io/projected/a3929aa3-2abd-489c-b7fd-c80e982db8e8-kube-api-access-26c2w\") pod \"node-ca-6jpkg\" (UID: \"a3929aa3-2abd-489c-b7fd-c80e982db8e8\") " pod="openshift-image-registry/node-ca-6jpkg" Jul 10 18:17:25.623088 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.623069 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z27zs\" (UniqueName: \"kubernetes.io/projected/517ac3e6-81be-4bbf-adb4-0cd53dfbf79a-kube-api-access-z27zs\") pod \"iptables-alerter-9d2qb\" (UID: \"517ac3e6-81be-4bbf-adb4-0cd53dfbf79a\") " pod="openshift-network-operator/iptables-alerter-9d2qb" Jul 10 18:17:25.623160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.623140 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckckc\" (UniqueName: \"kubernetes.io/projected/98c32dab-c8e9-489c-bbaf-0b46d8b0afa9-kube-api-access-ckckc\") pod \"multus-zdlvx\" (UID: \"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9\") " pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.624359 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.624226 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6pgk\" (UniqueName: \"kubernetes.io/projected/0bc69439-7cb2-4cda-8a15-a1904bc12785-kube-api-access-z6pgk\") pod \"tuned-9gx7f\" (UID: \"0bc69439-7cb2-4cda-8a15-a1904bc12785\") " pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.672687 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.672642 2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:17:25.689342 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.689280 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-54.ec2.internal" event={"ID":"b69d880dba39cd41f6208dd45b48e550","Type":"ContainerStarted","Data":"5ed0af85fff0c59bc0b321d21825d32243e8d6effcd0e57c446f3ab7454cd62c"} Jul 10 18:17:25.689523 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.689495 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-host-slash\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.689590 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.689535 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrmqx\" (UniqueName: \"kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx\") pod \"network-check-target-lftkj\" (UID: \"0a36554d-7c38-4893-9dca-ceac16436f2e\") " pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:25.689590 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.689564 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-host-run-ovn-kubernetes\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.689704 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.689588 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs\") pod \"network-metrics-daemon-ndsqx\" (UID: \"29ea8cc9-50a4-4990-a85c-b8c47aa81f0b\") " pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:25.689704 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.689633 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-host-run-netns\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.689704 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.689657 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-node-log\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.689704 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.689698 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05e2da1a-b49d-47e5-85e8-2ce1d30d3aae-tmp-dir\") pod \"node-resolver-kt69c\" (UID: \"05e2da1a-b49d-47e5-85e8-2ce1d30d3aae\") " pod="openshift-dns/node-resolver-kt69c" Jul 10 18:17:25.689884 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.689723 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-os-release\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.689884 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.689744 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-host-run-ovn-kubernetes\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.689884 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.689801 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-host-kubelet\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.689884 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.689843 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-host-slash\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.689884 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:25.689854 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:25.690099 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:25.689949 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs podName:29ea8cc9-50a4-4990-a85c-b8c47aa81f0b nodeName:}" failed. No retries permitted until 2026-07-10 18:17:26.189928572 +0000 UTC m=+3.089197674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs") pod "network-metrics-daemon-ndsqx" (UID: "29ea8cc9-50a4-4990-a85c-b8c47aa81f0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:25.690158 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.689750 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-host-kubelet\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.690158 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690119 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-host-run-netns\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.690245 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690156 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/05e2da1a-b49d-47e5-85e8-2ce1d30d3aae-hosts-file\") pod \"node-resolver-kt69c\" (UID: \"05e2da1a-b49d-47e5-85e8-2ce1d30d3aae\") " pod="openshift-dns/node-resolver-kt69c" Jul 10 18:17:25.690245 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690180 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-node-log\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.690245 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690187 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pt97\" (UniqueName: \"kubernetes.io/projected/05e2da1a-b49d-47e5-85e8-2ce1d30d3aae-kube-api-access-2pt97\") pod \"node-resolver-kt69c\" (UID: \"05e2da1a-b49d-47e5-85e8-2ce1d30d3aae\") " pod="openshift-dns/node-resolver-kt69c" Jul 10 18:17:25.690245 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690219 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-var-lib-openvswitch\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.690429 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690247 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-run-ovn\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.690429 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690270 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-log-socket\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.690429 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690296 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.690429 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690320 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdgg7\" (UniqueName: \"kubernetes.io/projected/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-kube-api-access-rdgg7\") pod \"network-metrics-daemon-ndsqx\" (UID: \"29ea8cc9-50a4-4990-a85c-b8c47aa81f0b\") " pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:25.690429 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690348 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-host-cni-bin\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.690429 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690379 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0cafad48-3998-465d-b4c8-b5d94cac6b3c-ovnkube-config\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.690429 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690403 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-cnibin\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.690429 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690429 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-run-openvswitch\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.690813 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690451 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0cafad48-3998-465d-b4c8-b5d94cac6b3c-env-overrides\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.690813 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690474 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0cafad48-3998-465d-b4c8-b5d94cac6b3c-ovnkube-script-lib\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.690813 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690501 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.690813 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690527 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.690813 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690556 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-host-cni-netd\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.690813 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690580 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0cafad48-3998-465d-b4c8-b5d94cac6b3c-ovn-node-metrics-cert\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.690813 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690627 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-etc-openvswitch\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.690813 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690655 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-cni-binary-copy\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.690813 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690662 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05e2da1a-b49d-47e5-85e8-2ce1d30d3aae-tmp-dir\") pod \"node-resolver-kt69c\" (UID: \"05e2da1a-b49d-47e5-85e8-2ce1d30d3aae\") " pod="openshift-dns/node-resolver-kt69c" Jul 10 18:17:25.690813 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690681 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.690813 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690754 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-os-release\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.690813 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690785 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-run-systemd\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.690813 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690815 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-systemd-units\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.691386 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690842 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6j4m\" (UniqueName: \"kubernetes.io/projected/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-kube-api-access-j6j4m\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.691386 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690872 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvdsv\" (UniqueName: \"kubernetes.io/projected/0cafad48-3998-465d-b4c8-b5d94cac6b3c-kube-api-access-pvdsv\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.691386 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690897 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-system-cni-dir\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.691386 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.690978 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-system-cni-dir\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.691386 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.691019 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-run-systemd\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.691386 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.691053 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-systemd-units\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.691386 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.691258 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.691386 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.691330 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/05e2da1a-b49d-47e5-85e8-2ce1d30d3aae-hosts-file\") pod \"node-resolver-kt69c\" (UID: \"05e2da1a-b49d-47e5-85e8-2ce1d30d3aae\") " pod="openshift-dns/node-resolver-kt69c" Jul 10 18:17:25.691766 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.691601 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-run-openvswitch\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.691766 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.691608 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-var-lib-openvswitch\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.691766 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.691717 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-log-socket\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.691907 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.691761 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-run-ovn\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.691907 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.691817 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-host-cni-netd\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.691907 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.691810 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.692047 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.692000 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.692103 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.692052 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-etc-openvswitch\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.692171 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.692148 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0cafad48-3998-465d-b4c8-b5d94cac6b3c-env-overrides\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.692222 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.692210 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0cafad48-3998-465d-b4c8-b5d94cac6b3c-host-cni-bin\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.692272 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.692221 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0cafad48-3998-465d-b4c8-b5d94cac6b3c-ovnkube-script-lib\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.692318 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.692297 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.692373 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.692354 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-cnibin\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.692698 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.692676 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0cafad48-3998-465d-b4c8-b5d94cac6b3c-ovnkube-config\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.692800 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.692699 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-cni-binary-copy\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.693018 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.692991 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-54.ec2.internal" event={"ID":"33a81504b2c888f51c5efff3d00523ac","Type":"ContainerStarted","Data":"53cf6f36442bb17200def9d7e966c3ef7f495b947e0f476f8b6ca893f7ec3326"} Jul 10 18:17:25.695574 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.695508 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0cafad48-3998-465d-b4c8-b5d94cac6b3c-ovn-node-metrics-cert\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.696303 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:25.696010 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:17:25.696303 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:25.696037 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:17:25.696303 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:25.696050 2571 projected.go:196] Error preparing data for projected volume kube-api-access-wrmqx for pod openshift-network-diagnostics/network-check-target-lftkj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:25.696303 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:25.696112 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx podName:0a36554d-7c38-4893-9dca-ceac16436f2e nodeName:}" failed. No retries permitted until 2026-07-10 18:17:26.196094603 +0000 UTC m=+3.095363710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wrmqx" (UniqueName: "kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx") pod "network-check-target-lftkj" (UID: "0a36554d-7c38-4893-9dca-ceac16436f2e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:25.702632 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.702547 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6j4m\" (UniqueName: \"kubernetes.io/projected/6058e5b7-e082-4aff-a0b2-1a46422bdb3a-kube-api-access-j6j4m\") pod \"multus-additional-cni-plugins-fzc82\" (UID: \"6058e5b7-e082-4aff-a0b2-1a46422bdb3a\") " pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.703343 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.703314 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pt97\" (UniqueName: \"kubernetes.io/projected/05e2da1a-b49d-47e5-85e8-2ce1d30d3aae-kube-api-access-2pt97\") pod \"node-resolver-kt69c\" (UID: \"05e2da1a-b49d-47e5-85e8-2ce1d30d3aae\") " pod="openshift-dns/node-resolver-kt69c" Jul 10 18:17:25.704767 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.704742 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvdsv\" (UniqueName: \"kubernetes.io/projected/0cafad48-3998-465d-b4c8-b5d94cac6b3c-kube-api-access-pvdsv\") pod \"ovnkube-node-7r9vz\" (UID: \"0cafad48-3998-465d-b4c8-b5d94cac6b3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.704981 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.704960 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdgg7\" (UniqueName: \"kubernetes.io/projected/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-kube-api-access-rdgg7\") pod \"network-metrics-daemon-ndsqx\" (UID: \"29ea8cc9-50a4-4990-a85c-b8c47aa81f0b\") " pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:25.775605 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.775358 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hn6g2" Jul 10 18:17:25.784774 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.784744 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" Jul 10 18:17:25.793519 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.793493 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zdlvx" Jul 10 18:17:25.801606 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.801574 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" Jul 10 18:17:25.823538 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.823496 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6jpkg" Jul 10 18:17:25.838731 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.838697 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9d2qb" Jul 10 18:17:25.846885 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.846853 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:25.854939 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.854909 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fzc82" Jul 10 18:17:25.862757 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:25.862728 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kt69c" Jul 10 18:17:26.094483 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:26.094391 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret\") pod \"global-pull-secret-syncer-9rqql\" (UID: \"91c02392-14bc-403d-ab9f-131ed9213d28\") " pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:26.094684 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:26.094538 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:26.094684 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:26.094609 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret podName:91c02392-14bc-403d-ab9f-131ed9213d28 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:27.094589011 +0000 UTC m=+3.993858100 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret") pod "global-pull-secret-syncer-9rqql" (UID: "91c02392-14bc-403d-ab9f-131ed9213d28") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:26.194949 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:26.194911 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs\") pod \"network-metrics-daemon-ndsqx\" (UID: \"29ea8cc9-50a4-4990-a85c-b8c47aa81f0b\") " pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:26.195145 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:26.195049 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:26.195145 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:26.195107 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs podName:29ea8cc9-50a4-4990-a85c-b8c47aa81f0b nodeName:}" failed. No retries permitted until 2026-07-10 18:17:27.19509263 +0000 UTC m=+4.094361719 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs") pod "network-metrics-daemon-ndsqx" (UID: "29ea8cc9-50a4-4990-a85c-b8c47aa81f0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:26.224927 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:26.224895 2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 18:17:26.295465 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:26.295429 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrmqx\" (UniqueName: \"kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx\") pod \"network-check-target-lftkj\" (UID: \"0a36554d-7c38-4893-9dca-ceac16436f2e\") " pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:26.295667 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:26.295607 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:17:26.295667 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:26.295648 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:17:26.295667 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:26.295658 2571 projected.go:196] Error preparing data for projected volume kube-api-access-wrmqx for pod openshift-network-diagnostics/network-check-target-lftkj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:26.295793 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:26.295717 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx podName:0a36554d-7c38-4893-9dca-ceac16436f2e nodeName:}" failed. No retries permitted until 2026-07-10 18:17:27.295700139 +0000 UTC m=+4.194969238 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wrmqx" (UniqueName: "kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx") pod "network-check-target-lftkj" (UID: "0a36554d-7c38-4893-9dca-ceac16436f2e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:26.514065 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:26.514006 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 18:12:25 +0000 UTC" deadline="2028-04-23 23:51:27.860178218 +0000 UTC" Jul 10 18:17:26.514065 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:26.514060 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15677h34m1.346121792s" Jul 10 18:17:26.677911 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:26.677879 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:26.678312 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:26.678067 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsqx" podUID="29ea8cc9-50a4-4990-a85c-b8c47aa81f0b" Jul 10 18:17:26.678312 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:26.678109 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:26.678312 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:26.678166 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lftkj" podUID="0a36554d-7c38-4893-9dca-ceac16436f2e" Jul 10 18:17:26.790078 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:26.789838 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fbdae78_1e8a_4b53_aff0_0f807312bd7e.slice/crio-70ac350baadae399dcd71d6dd0073274450e765b164a2dbf7e720433bf19bd63 WatchSource:0}: Error finding container 70ac350baadae399dcd71d6dd0073274450e765b164a2dbf7e720433bf19bd63: Status 404 returned error can't find the container with id 70ac350baadae399dcd71d6dd0073274450e765b164a2dbf7e720433bf19bd63 Jul 10 18:17:26.791608 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:26.791577 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3929aa3_2abd_489c_b7fd_c80e982db8e8.slice/crio-1b03640db7e5a31e13923f65fbe9f5961910c9b8ee8170c23a7c7b028d9c6ce1 WatchSource:0}: Error finding container 1b03640db7e5a31e13923f65fbe9f5961910c9b8ee8170c23a7c7b028d9c6ce1: Status 404 returned error can't find the container with id 1b03640db7e5a31e13923f65fbe9f5961910c9b8ee8170c23a7c7b028d9c6ce1 Jul 10 18:17:26.792260 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:26.792222 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cafad48_3998_465d_b4c8_b5d94cac6b3c.slice/crio-6aae9983231444d6a39444523709b5db4df3fed655dc8a71462f5d37cc530592 WatchSource:0}: Error finding container 6aae9983231444d6a39444523709b5db4df3fed655dc8a71462f5d37cc530592: Status 404 returned error can't find the container with id 6aae9983231444d6a39444523709b5db4df3fed655dc8a71462f5d37cc530592 Jul 10 18:17:26.794576 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:26.794293 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6058e5b7_e082_4aff_a0b2_1a46422bdb3a.slice/crio-165e6f0744c02e60052918602bd5bc0e14981abe4e2b58011c672467b4c28818 WatchSource:0}: Error finding container 165e6f0744c02e60052918602bd5bc0e14981abe4e2b58011c672467b4c28818: Status 404 returned error can't find the container with id 165e6f0744c02e60052918602bd5bc0e14981abe4e2b58011c672467b4c28818 Jul 10 18:17:26.795082 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:26.795055 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod517ac3e6_81be_4bbf_adb4_0cd53dfbf79a.slice/crio-f236d783c787766f1ba8b164d9c0913470a2b39cb5bc4b6233f35228515e2497 WatchSource:0}: Error finding container f236d783c787766f1ba8b164d9c0913470a2b39cb5bc4b6233f35228515e2497: Status 404 returned error can't find the container with id f236d783c787766f1ba8b164d9c0913470a2b39cb5bc4b6233f35228515e2497 Jul 10 18:17:26.796264 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:26.796128 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bc69439_7cb2_4cda_8a15_a1904bc12785.slice/crio-2d38a43fe6b821c3281f085bac492fcca94829e83c4482bd57d0cda631c5b739 WatchSource:0}: Error finding container 2d38a43fe6b821c3281f085bac492fcca94829e83c4482bd57d0cda631c5b739: Status 404 returned error can't find the container with id 2d38a43fe6b821c3281f085bac492fcca94829e83c4482bd57d0cda631c5b739 Jul 10 18:17:26.800428 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:26.800386 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05e2da1a_b49d_47e5_85e8_2ce1d30d3aae.slice/crio-5b6444c643dc498a8982494d29d105b0054270a3b51842524fac69c4fe4fdcf3 WatchSource:0}: Error finding container 5b6444c643dc498a8982494d29d105b0054270a3b51842524fac69c4fe4fdcf3: Status 404 returned error can't find the container with id 5b6444c643dc498a8982494d29d105b0054270a3b51842524fac69c4fe4fdcf3 Jul 10 18:17:26.806873 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:26.806459 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod628f9bd4_aa72_4a8d_a8ad_31ddd10e208c.slice/crio-506e817c498d9eff3e349c5eaf5e9765a463d43b97989b25fa9d466507a2f4af WatchSource:0}: Error finding container 506e817c498d9eff3e349c5eaf5e9765a463d43b97989b25fa9d466507a2f4af: Status 404 returned error can't find the container with id 506e817c498d9eff3e349c5eaf5e9765a463d43b97989b25fa9d466507a2f4af Jul 10 18:17:27.101245 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:27.101128 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret\") pod \"global-pull-secret-syncer-9rqql\" (UID: \"91c02392-14bc-403d-ab9f-131ed9213d28\") " pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:27.101245 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:27.101238 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:27.101425 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:27.101292 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret podName:91c02392-14bc-403d-ab9f-131ed9213d28 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:29.101276783 +0000 UTC m=+6.000545869 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret") pod "global-pull-secret-syncer-9rqql" (UID: "91c02392-14bc-403d-ab9f-131ed9213d28") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:27.201930 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:27.201843 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs\") pod \"network-metrics-daemon-ndsqx\" (UID: \"29ea8cc9-50a4-4990-a85c-b8c47aa81f0b\") " pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:27.202113 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:27.201994 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:27.202113 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:27.202075 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs podName:29ea8cc9-50a4-4990-a85c-b8c47aa81f0b nodeName:}" failed. No retries permitted until 2026-07-10 18:17:29.202050861 +0000 UTC m=+6.101319953 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs") pod "network-metrics-daemon-ndsqx" (UID: "29ea8cc9-50a4-4990-a85c-b8c47aa81f0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:27.302448 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:27.302390 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrmqx\" (UniqueName: \"kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx\") pod \"network-check-target-lftkj\" (UID: \"0a36554d-7c38-4893-9dca-ceac16436f2e\") " pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:27.302661 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:27.302608 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:17:27.302661 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:27.302647 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:17:27.302661 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:27.302661 2571 projected.go:196] Error preparing data for projected volume kube-api-access-wrmqx for pod openshift-network-diagnostics/network-check-target-lftkj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:27.302854 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:27.302722 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx podName:0a36554d-7c38-4893-9dca-ceac16436f2e nodeName:}" failed. No retries permitted until 2026-07-10 18:17:29.302703963 +0000 UTC m=+6.201973068 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wrmqx" (UniqueName: "kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx") pod "network-check-target-lftkj" (UID: "0a36554d-7c38-4893-9dca-ceac16436f2e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:27.515095 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:27.515025 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 18:12:25 +0000 UTC" deadline="2027-12-13 12:52:39.655462579 +0000 UTC" Jul 10 18:17:27.515095 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:27.515093 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12498h35m12.140374278s" Jul 10 18:17:27.680516 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:27.679963 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:27.680516 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:27.680097 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9rqql" podUID="91c02392-14bc-403d-ab9f-131ed9213d28" Jul 10 18:17:27.704710 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:27.704655 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" event={"ID":"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c","Type":"ContainerStarted","Data":"506e817c498d9eff3e349c5eaf5e9765a463d43b97989b25fa9d466507a2f4af"} Jul 10 18:17:27.719544 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:27.719387 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kt69c" event={"ID":"05e2da1a-b49d-47e5-85e8-2ce1d30d3aae","Type":"ContainerStarted","Data":"5b6444c643dc498a8982494d29d105b0054270a3b51842524fac69c4fe4fdcf3"} Jul 10 18:17:27.731822 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:27.731783 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9d2qb" event={"ID":"517ac3e6-81be-4bbf-adb4-0cd53dfbf79a","Type":"ContainerStarted","Data":"f236d783c787766f1ba8b164d9c0913470a2b39cb5bc4b6233f35228515e2497"} Jul 10 18:17:27.743764 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:27.743724 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6jpkg" event={"ID":"a3929aa3-2abd-489c-b7fd-c80e982db8e8","Type":"ContainerStarted","Data":"1b03640db7e5a31e13923f65fbe9f5961910c9b8ee8170c23a7c7b028d9c6ce1"} Jul 10 18:17:27.749063 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:27.748965 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hn6g2" event={"ID":"3fbdae78-1e8a-4b53-aff0-0f807312bd7e","Type":"ContainerStarted","Data":"70ac350baadae399dcd71d6dd0073274450e765b164a2dbf7e720433bf19bd63"} Jul 10 18:17:27.756089 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:27.756050 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-54.ec2.internal" event={"ID":"33a81504b2c888f51c5efff3d00523ac","Type":"ContainerStarted","Data":"e16c2eb56554cedf2c62932876b4653c9b320fcb2716364974dcfa18745090b3"} Jul 10 18:17:27.780941 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:27.780900 2571 generic.go:356] "Generic (PLEG): container finished" podID="b69d880dba39cd41f6208dd45b48e550" containerID="09920a1a1f2c3413b6c059003944e06d01983c7370c6ecfcaa7b8a39e9ca6edb" exitCode=0 Jul 10 18:17:27.781155 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:27.781031 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-54.ec2.internal" event={"ID":"b69d880dba39cd41f6208dd45b48e550","Type":"ContainerDied","Data":"09920a1a1f2c3413b6c059003944e06d01983c7370c6ecfcaa7b8a39e9ca6edb"} Jul 10 18:17:27.791425 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:27.791295 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zdlvx" event={"ID":"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9","Type":"ContainerStarted","Data":"a91cb2c102721043285790ddfb36ff8a8e29a093bd0d2375e007051c4f6d2cd0"} Jul 10 18:17:27.795465 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:27.795429 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" event={"ID":"0bc69439-7cb2-4cda-8a15-a1904bc12785","Type":"ContainerStarted","Data":"2d38a43fe6b821c3281f085bac492fcca94829e83c4482bd57d0cda631c5b739"} Jul 10 18:17:27.796852 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:27.796796 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-54.ec2.internal" podStartSLOduration=2.796776249 podStartE2EDuration="2.796776249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:17:27.769594427 +0000 UTC m=+4.668863536" watchObservedRunningTime="2026-07-10 18:17:27.796776249 +0000 UTC m=+4.696045358" Jul 10 18:17:27.821162 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:27.821013 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzc82" event={"ID":"6058e5b7-e082-4aff-a0b2-1a46422bdb3a","Type":"ContainerStarted","Data":"165e6f0744c02e60052918602bd5bc0e14981abe4e2b58011c672467b4c28818"} Jul 10 18:17:27.841627 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:27.841485 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" event={"ID":"0cafad48-3998-465d-b4c8-b5d94cac6b3c","Type":"ContainerStarted","Data":"6aae9983231444d6a39444523709b5db4df3fed655dc8a71462f5d37cc530592"} Jul 10 18:17:28.678850 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:28.678067 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:28.678850 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:28.678216 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsqx" podUID="29ea8cc9-50a4-4990-a85c-b8c47aa81f0b" Jul 10 18:17:28.678850 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:28.678646 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:28.678850 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:28.678748 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lftkj" podUID="0a36554d-7c38-4893-9dca-ceac16436f2e" Jul 10 18:17:28.851610 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:28.851420 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-54.ec2.internal" event={"ID":"b69d880dba39cd41f6208dd45b48e550","Type":"ContainerStarted","Data":"0d166b3a471a9c492b2d763ddbfa3ae7569107ae8d04af1dc241af3731e77100"} Jul 10 18:17:28.870899 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:28.870834 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-54.ec2.internal" podStartSLOduration=3.8708065830000002 podStartE2EDuration="3.870806583s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:17:28.869364345 +0000 UTC m=+5.768633468" watchObservedRunningTime="2026-07-10 18:17:28.870806583 +0000 UTC m=+5.770075692" Jul 10 18:17:29.120128 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:29.119442 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret\") pod \"global-pull-secret-syncer-9rqql\" (UID: \"91c02392-14bc-403d-ab9f-131ed9213d28\") " pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:29.120128 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:29.119686 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:29.120128 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:29.119754 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret podName:91c02392-14bc-403d-ab9f-131ed9213d28 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:33.11973485 +0000 UTC m=+10.019003939 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret") pod "global-pull-secret-syncer-9rqql" (UID: "91c02392-14bc-403d-ab9f-131ed9213d28") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:29.221430 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:29.220861 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs\") pod \"network-metrics-daemon-ndsqx\" (UID: \"29ea8cc9-50a4-4990-a85c-b8c47aa81f0b\") " pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:29.221430 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:29.221013 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:29.221430 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:29.221080 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs podName:29ea8cc9-50a4-4990-a85c-b8c47aa81f0b nodeName:}" failed. No retries permitted until 2026-07-10 18:17:33.221059343 +0000 UTC m=+10.120328435 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs") pod "network-metrics-daemon-ndsqx" (UID: "29ea8cc9-50a4-4990-a85c-b8c47aa81f0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:29.322251 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:29.321564 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrmqx\" (UniqueName: \"kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx\") pod \"network-check-target-lftkj\" (UID: \"0a36554d-7c38-4893-9dca-ceac16436f2e\") " pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:29.322251 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:29.321789 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:17:29.322251 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:29.321810 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:17:29.322251 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:29.321823 2571 projected.go:196] Error preparing data for projected volume kube-api-access-wrmqx for pod openshift-network-diagnostics/network-check-target-lftkj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:29.322251 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:29.321880 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx podName:0a36554d-7c38-4893-9dca-ceac16436f2e nodeName:}" failed. No retries permitted until 2026-07-10 18:17:33.321861929 +0000 UTC m=+10.221131040 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wrmqx" (UniqueName: "kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx") pod "network-check-target-lftkj" (UID: "0a36554d-7c38-4893-9dca-ceac16436f2e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:29.680230 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:29.679737 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:29.680230 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:29.679867 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9rqql" podUID="91c02392-14bc-403d-ab9f-131ed9213d28" Jul 10 18:17:30.677555 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:30.677513 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:30.678064 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:30.677684 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsqx" podUID="29ea8cc9-50a4-4990-a85c-b8c47aa81f0b" Jul 10 18:17:30.678064 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:30.677780 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:30.678064 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:30.677888 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lftkj" podUID="0a36554d-7c38-4893-9dca-ceac16436f2e" Jul 10 18:17:31.678236 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:31.678148 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:31.678737 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:31.678289 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9rqql" podUID="91c02392-14bc-403d-ab9f-131ed9213d28" Jul 10 18:17:32.677894 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:32.677860 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:32.678076 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:32.677997 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lftkj" podUID="0a36554d-7c38-4893-9dca-ceac16436f2e" Jul 10 18:17:32.678548 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:32.677860 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:32.678548 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:32.678477 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsqx" podUID="29ea8cc9-50a4-4990-a85c-b8c47aa81f0b" Jul 10 18:17:33.155781 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:33.155660 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret\") pod \"global-pull-secret-syncer-9rqql\" (UID: \"91c02392-14bc-403d-ab9f-131ed9213d28\") " pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:33.155957 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:33.155873 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:33.156027 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:33.155959 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret podName:91c02392-14bc-403d-ab9f-131ed9213d28 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:41.155937041 +0000 UTC m=+18.055206150 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret") pod "global-pull-secret-syncer-9rqql" (UID: "91c02392-14bc-403d-ab9f-131ed9213d28") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:33.256865 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:33.256828 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs\") pod \"network-metrics-daemon-ndsqx\" (UID: \"29ea8cc9-50a4-4990-a85c-b8c47aa81f0b\") " pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:33.257058 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:33.256976 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:33.257058 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:33.257026 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs podName:29ea8cc9-50a4-4990-a85c-b8c47aa81f0b nodeName:}" failed. No retries permitted until 2026-07-10 18:17:41.257013828 +0000 UTC m=+18.156282914 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs") pod "network-metrics-daemon-ndsqx" (UID: "29ea8cc9-50a4-4990-a85c-b8c47aa81f0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:33.357784 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:33.357398 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrmqx\" (UniqueName: \"kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx\") pod \"network-check-target-lftkj\" (UID: \"0a36554d-7c38-4893-9dca-ceac16436f2e\") " pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:33.357784 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:33.357638 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:17:33.357784 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:33.357658 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:17:33.357784 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:33.357671 2571 projected.go:196] Error preparing data for projected volume kube-api-access-wrmqx for pod openshift-network-diagnostics/network-check-target-lftkj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:33.357784 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:33.357732 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx podName:0a36554d-7c38-4893-9dca-ceac16436f2e nodeName:}" failed. No retries permitted until 2026-07-10 18:17:41.357711969 +0000 UTC m=+18.256981057 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wrmqx" (UniqueName: "kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx") pod "network-check-target-lftkj" (UID: "0a36554d-7c38-4893-9dca-ceac16436f2e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:33.679567 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:33.679036 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:33.679567 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:33.679177 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9rqql" podUID="91c02392-14bc-403d-ab9f-131ed9213d28" Jul 10 18:17:34.677525 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:34.677430 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:34.677525 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:34.677462 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:34.677805 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:34.677581 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsqx" podUID="29ea8cc9-50a4-4990-a85c-b8c47aa81f0b" Jul 10 18:17:34.678077 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:34.678008 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lftkj" podUID="0a36554d-7c38-4893-9dca-ceac16436f2e" Jul 10 18:17:35.681480 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:35.681431 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:35.681970 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:35.681570 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9rqql" podUID="91c02392-14bc-403d-ab9f-131ed9213d28" Jul 10 18:17:36.677488 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:36.677448 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:36.677488 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:36.677485 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:36.678151 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:36.677602 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsqx" podUID="29ea8cc9-50a4-4990-a85c-b8c47aa81f0b" Jul 10 18:17:36.678151 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:36.677720 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lftkj" podUID="0a36554d-7c38-4893-9dca-ceac16436f2e" Jul 10 18:17:37.677870 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:37.677789 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:37.678267 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:37.677920 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9rqql" podUID="91c02392-14bc-403d-ab9f-131ed9213d28" Jul 10 18:17:38.677374 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:38.677338 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:38.677573 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:38.677465 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsqx" podUID="29ea8cc9-50a4-4990-a85c-b8c47aa81f0b" Jul 10 18:17:38.677573 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:38.677516 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:38.677707 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:38.677629 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lftkj" podUID="0a36554d-7c38-4893-9dca-ceac16436f2e" Jul 10 18:17:39.678102 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:39.678064 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:39.678554 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:39.678203 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9rqql" podUID="91c02392-14bc-403d-ab9f-131ed9213d28" Jul 10 18:17:40.677484 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:40.677443 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:40.677484 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:40.677484 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:40.677807 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:40.677605 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsqx" podUID="29ea8cc9-50a4-4990-a85c-b8c47aa81f0b" Jul 10 18:17:40.677807 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:40.677739 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lftkj" podUID="0a36554d-7c38-4893-9dca-ceac16436f2e" Jul 10 18:17:41.221653 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:41.221600 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret\") pod \"global-pull-secret-syncer-9rqql\" (UID: \"91c02392-14bc-403d-ab9f-131ed9213d28\") " pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:41.222025 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:41.221739 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:41.222025 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:41.221801 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret podName:91c02392-14bc-403d-ab9f-131ed9213d28 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:57.221785006 +0000 UTC m=+34.121054092 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret") pod "global-pull-secret-syncer-9rqql" (UID: "91c02392-14bc-403d-ab9f-131ed9213d28") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:41.322723 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:41.322686 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs\") pod \"network-metrics-daemon-ndsqx\" (UID: \"29ea8cc9-50a4-4990-a85c-b8c47aa81f0b\") " pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:41.322887 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:41.322814 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:41.322887 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:41.322870 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs podName:29ea8cc9-50a4-4990-a85c-b8c47aa81f0b nodeName:}" failed. No retries permitted until 2026-07-10 18:17:57.322851506 +0000 UTC m=+34.222120594 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs") pod "network-metrics-daemon-ndsqx" (UID: "29ea8cc9-50a4-4990-a85c-b8c47aa81f0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:41.423732 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:41.423694 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrmqx\" (UniqueName: \"kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx\") pod \"network-check-target-lftkj\" (UID: \"0a36554d-7c38-4893-9dca-ceac16436f2e\") " pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:41.423935 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:41.423858 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:17:41.423935 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:41.423880 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:17:41.423935 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:41.423892 2571 projected.go:196] Error preparing data for projected volume kube-api-access-wrmqx for pod openshift-network-diagnostics/network-check-target-lftkj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:41.424092 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:41.423948 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx podName:0a36554d-7c38-4893-9dca-ceac16436f2e nodeName:}" failed. No retries permitted until 2026-07-10 18:17:57.423934288 +0000 UTC m=+34.323203374 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wrmqx" (UniqueName: "kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx") pod "network-check-target-lftkj" (UID: "0a36554d-7c38-4893-9dca-ceac16436f2e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:41.678153 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:41.678105 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:41.678347 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:41.678249 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9rqql" podUID="91c02392-14bc-403d-ab9f-131ed9213d28" Jul 10 18:17:42.677918 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:42.677871 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:42.677918 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:42.677918 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:42.678466 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:42.678033 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsqx" podUID="29ea8cc9-50a4-4990-a85c-b8c47aa81f0b" Jul 10 18:17:42.678466 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:42.678173 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lftkj" podUID="0a36554d-7c38-4893-9dca-ceac16436f2e" Jul 10 18:17:43.679094 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:43.679052 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:43.679560 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:43.679179 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9rqql" podUID="91c02392-14bc-403d-ab9f-131ed9213d28" Jul 10 18:17:44.677626 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:44.677577 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:44.677804 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:44.677634 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:44.677804 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:44.677720 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lftkj" podUID="0a36554d-7c38-4893-9dca-ceac16436f2e" Jul 10 18:17:44.677933 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:44.677816 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsqx" podUID="29ea8cc9-50a4-4990-a85c-b8c47aa81f0b" Jul 10 18:17:45.677867 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:45.677426 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:45.677867 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:45.677822 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9rqql" podUID="91c02392-14bc-403d-ab9f-131ed9213d28" Jul 10 18:17:45.886464 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:45.886431 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" event={"ID":"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c","Type":"ContainerStarted","Data":"3917df81e93ffde9b61f733ea1b0b2e854a80cca8fd36976cdfc8041dde881e0"} Jul 10 18:17:45.887792 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:45.887764 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kt69c" event={"ID":"05e2da1a-b49d-47e5-85e8-2ce1d30d3aae","Type":"ContainerStarted","Data":"21e65805797c26118373e5d2c0076e19e38e9af86b23f23c94d8bd5c7b736ab7"} Jul 10 18:17:45.889139 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:45.889117 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6jpkg" event={"ID":"a3929aa3-2abd-489c-b7fd-c80e982db8e8","Type":"ContainerStarted","Data":"d9aa2aa50ed13f36189ea6f14c29a4220b8fe51a11a69420003395e181a7ede5"} Jul 10 18:17:45.890391 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:45.890363 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hn6g2" event={"ID":"3fbdae78-1e8a-4b53-aff0-0f807312bd7e","Type":"ContainerStarted","Data":"72b89dda54fb4b526e38284e0245fd644ac7219169a043df71c5a07ef76dd4b7"} Jul 10 18:17:45.891799 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:45.891772 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zdlvx" event={"ID":"98c32dab-c8e9-489c-bbaf-0b46d8b0afa9","Type":"ContainerStarted","Data":"571d7f4c57e1678f059c4edc07b4de372b2717a6fb17090c81c37c79beb23676"} Jul 10 18:17:45.893189 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:45.893170 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" event={"ID":"0bc69439-7cb2-4cda-8a15-a1904bc12785","Type":"ContainerStarted","Data":"ddbd001575a88dc8a145548de58174ee1cfc0cfef62237c07fafabbe64ae5613"} Jul 10 18:17:45.894627 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:45.894592 2571 generic.go:356] "Generic (PLEG): container finished" podID="6058e5b7-e082-4aff-a0b2-1a46422bdb3a" containerID="b73c5b48d453b8d3e365c92c6a818bd181ce866bdb1f3d19416115c882b63688" exitCode=0 Jul 10 18:17:45.894717 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:45.894650 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzc82" event={"ID":"6058e5b7-e082-4aff-a0b2-1a46422bdb3a","Type":"ContainerDied","Data":"b73c5b48d453b8d3e365c92c6a818bd181ce866bdb1f3d19416115c882b63688"} Jul 10 18:17:45.896153 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:45.896140 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r9vz_0cafad48-3998-465d-b4c8-b5d94cac6b3c/ovn-acl-logging/0.log" Jul 10 18:17:45.896548 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:45.896523 2571 generic.go:356] "Generic (PLEG): container finished" podID="0cafad48-3998-465d-b4c8-b5d94cac6b3c" containerID="455857d3b41ee61f59538e749df22f45ad8a4cc56c262f45aed4ce1e5149042b" exitCode=1 Jul 10 18:17:45.896652 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:45.896552 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" event={"ID":"0cafad48-3998-465d-b4c8-b5d94cac6b3c","Type":"ContainerDied","Data":"455857d3b41ee61f59538e749df22f45ad8a4cc56c262f45aed4ce1e5149042b"} Jul 10 18:17:45.896652 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:45.896632 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" event={"ID":"0cafad48-3998-465d-b4c8-b5d94cac6b3c","Type":"ContainerStarted","Data":"bfef17d60d0b3179c9f0904d2509341207644ebb87cdb2c46f6659738ffe3f5f"} Jul 10 18:17:45.904509 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:45.904452 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kt69c" podStartSLOduration=4.261591309 podStartE2EDuration="22.904439064s" totalImagesPullingTime="18.642847755s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:17:45.903911645 +0000 UTC m=+22.803180777" watchObservedRunningTime="2026-07-10 18:17:45.904439064 +0000 UTC m=+22.803708171" Jul 10 18:17:45.945516 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:45.945443 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zdlvx" podStartSLOduration=4.247853352 podStartE2EDuration="22.945427004s" totalImagesPullingTime="18.697573652s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:17:45.944956899 +0000 UTC m=+22.844226007" watchObservedRunningTime="2026-07-10 18:17:45.945427004 +0000 UTC m=+22.844696111" Jul 10 18:17:45.960576 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:45.960397 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6jpkg" podStartSLOduration=4.300589156 podStartE2EDuration="22.960380286s" totalImagesPullingTime="18.65979113s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:17:45.960262953 +0000 UTC m=+22.859532061" watchObservedRunningTime="2026-07-10 18:17:45.960380286 +0000 UTC m=+22.859649391" Jul 10 18:17:45.979235 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:45.979181 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hn6g2" podStartSLOduration=12.324518340000001 podStartE2EDuration="22.979166058s" totalImagesPullingTime="10.654647718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:17:45.978907047 +0000 UTC m=+22.878176157" watchObservedRunningTime="2026-07-10 18:17:45.979166058 +0000 UTC m=+22.878435165" Jul 10 18:17:46.001115 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:46.001058 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9gx7f" podStartSLOduration=4.346773674 podStartE2EDuration="23.001041518s" totalImagesPullingTime="18.654267844s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:17:46.0007409 +0000 UTC m=+22.900010018" watchObservedRunningTime="2026-07-10 18:17:46.001041518 +0000 UTC m=+22.900310664" Jul 10 18:17:46.677666 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:46.677607 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:46.677890 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:46.677757 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsqx" podUID="29ea8cc9-50a4-4990-a85c-b8c47aa81f0b" Jul 10 18:17:46.677890 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:46.677816 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:46.678400 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:46.677925 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lftkj" podUID="0a36554d-7c38-4893-9dca-ceac16436f2e" Jul 10 18:17:46.902478 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:46.902451 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r9vz_0cafad48-3998-465d-b4c8-b5d94cac6b3c/ovn-acl-logging/0.log" Jul 10 18:17:46.903051 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:46.903018 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" event={"ID":"0cafad48-3998-465d-b4c8-b5d94cac6b3c","Type":"ContainerStarted","Data":"7a9b62d69deed8c565f1d6932e5f9e18cf4a32eb599847a9c25f809f77e81d29"} Jul 10 18:17:46.903173 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:46.903112 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" event={"ID":"0cafad48-3998-465d-b4c8-b5d94cac6b3c","Type":"ContainerStarted","Data":"b0f094113b08654aa8137289f9b7b03dd74a052a9dbb9a4aea431b95d3aee949"} Jul 10 18:17:46.903173 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:46.903162 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" event={"ID":"0cafad48-3998-465d-b4c8-b5d94cac6b3c","Type":"ContainerStarted","Data":"dca9a227cd46be71618ca99f457a5603b583214642e0629b9f9cf595be96834e"} Jul 10 18:17:46.903279 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:46.903208 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" event={"ID":"0cafad48-3998-465d-b4c8-b5d94cac6b3c","Type":"ContainerStarted","Data":"7aa3afe72e5cef5e8fc7cd129ab5a9d71bdba285c3f55c33436cd940ec9c6611"} Jul 10 18:17:46.940475 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:46.940426 2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 18:17:47.550756 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:47.550386 2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T18:17:46.940451931Z","UUID":"e93c8bc8-2e41-4068-90de-f3eaa60933ee","Handler":null,"Name":"","Endpoint":""} Jul 10 18:17:47.552328 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:47.552303 2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 18:17:47.552472 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:47.552351 2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 18:17:47.678840 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:47.678363 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:47.678840 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:47.678506 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9rqql" podUID="91c02392-14bc-403d-ab9f-131ed9213d28" Jul 10 18:17:47.906917 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:47.906837 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" event={"ID":"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c","Type":"ContainerStarted","Data":"94301afd1a3001ddbda80fde92b4c3d83fbd323d3d5b2f151b3dfc8c03f4e72e"} Jul 10 18:17:47.908181 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:47.908146 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9d2qb" event={"ID":"517ac3e6-81be-4bbf-adb4-0cd53dfbf79a","Type":"ContainerStarted","Data":"1c6040cbba4a6cb2b6cfc89cd1deaaf7bacb950ebc774c9f4d5ae670151ad50b"} Jul 10 18:17:47.924577 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:47.924526 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9d2qb" podStartSLOduration=6.271482243 podStartE2EDuration="24.924510145s" totalImagesPullingTime="18.653027902s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:17:47.92386623 +0000 UTC m=+24.823135339" watchObservedRunningTime="2026-07-10 18:17:47.924510145 +0000 UTC m=+24.823779253" Jul 10 18:17:48.444050 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:48.444010 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hn6g2" Jul 10 18:17:48.444718 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:48.444694 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hn6g2" Jul 10 18:17:48.677535 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:48.677456 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:48.677735 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:48.677574 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lftkj" podUID="0a36554d-7c38-4893-9dca-ceac16436f2e" Jul 10 18:17:48.677735 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:48.677635 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:48.677735 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:48.677727 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsqx" podUID="29ea8cc9-50a4-4990-a85c-b8c47aa81f0b" Jul 10 18:17:48.916009 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:48.915983 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r9vz_0cafad48-3998-465d-b4c8-b5d94cac6b3c/ovn-acl-logging/0.log" Jul 10 18:17:48.916570 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:48.916395 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" event={"ID":"0cafad48-3998-465d-b4c8-b5d94cac6b3c","Type":"ContainerStarted","Data":"8ae9fdb13b97c5ccfdc41283759fe5a6150486ec1c24c5ddbe8a1b8195c308b7"} Jul 10 18:17:48.918931 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:48.918896 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" event={"ID":"628f9bd4-aa72-4a8d-a8ad-31ddd10e208c","Type":"ContainerStarted","Data":"d28f96002c504492e7922c8d25fb4ba64f856d454f468bd913ab8f73021449b9"} Jul 10 18:17:48.919382 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:48.919340 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hn6g2" Jul 10 18:17:48.919683 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:48.919664 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hn6g2" Jul 10 18:17:48.937219 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:48.937141 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z8kh8" podStartSLOduration=5.151693862 podStartE2EDuration="25.937127911s" totalImagesPullingTime="20.785434049s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:17:48.936590165 +0000 UTC m=+25.835859285" watchObservedRunningTime="2026-07-10 18:17:48.937127911 +0000 UTC m=+25.836397018" Jul 10 18:17:49.677431 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:49.677398 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:49.677653 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:49.677531 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9rqql" podUID="91c02392-14bc-403d-ab9f-131ed9213d28" Jul 10 18:17:50.678004 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:50.677831 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:50.678562 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:50.677876 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:50.678562 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:50.678092 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsqx" podUID="29ea8cc9-50a4-4990-a85c-b8c47aa81f0b" Jul 10 18:17:50.678562 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:50.678170 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lftkj" podUID="0a36554d-7c38-4893-9dca-ceac16436f2e" Jul 10 18:17:50.924895 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:50.924861 2571 generic.go:356] "Generic (PLEG): container finished" podID="6058e5b7-e082-4aff-a0b2-1a46422bdb3a" containerID="f31259424b8bb0d01520b20bb771a2c11aac5dc38d834db0ea99e6f8f064e46b" exitCode=0 Jul 10 18:17:50.925074 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:50.924950 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzc82" event={"ID":"6058e5b7-e082-4aff-a0b2-1a46422bdb3a","Type":"ContainerDied","Data":"f31259424b8bb0d01520b20bb771a2c11aac5dc38d834db0ea99e6f8f064e46b"} Jul 10 18:17:50.928515 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:50.928494 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r9vz_0cafad48-3998-465d-b4c8-b5d94cac6b3c/ovn-acl-logging/0.log" Jul 10 18:17:50.928890 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:50.928871 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" event={"ID":"0cafad48-3998-465d-b4c8-b5d94cac6b3c","Type":"ContainerStarted","Data":"460b2aa8d6fc4370fa7ef3d748111f87bc01b06b9429d57f00e6331a0accc7a0"} Jul 10 18:17:50.929191 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:50.929155 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:50.929309 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:50.929292 2571 scope.go:117] "RemoveContainer" containerID="455857d3b41ee61f59538e749df22f45ad8a4cc56c262f45aed4ce1e5149042b" Jul 10 18:17:50.948185 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:50.948165 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:51.677456 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:51.677375 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:51.677641 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:51.677489 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9rqql" podUID="91c02392-14bc-403d-ab9f-131ed9213d28" Jul 10 18:17:51.934787 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:51.934714 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r9vz_0cafad48-3998-465d-b4c8-b5d94cac6b3c/ovn-acl-logging/0.log" Jul 10 18:17:51.935266 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:51.935053 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" event={"ID":"0cafad48-3998-465d-b4c8-b5d94cac6b3c","Type":"ContainerStarted","Data":"788ff0b274d7534c09618e493a6e87994f90abbe5461409b87501188763cc14b"} Jul 10 18:17:51.935355 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:51.935335 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:51.935476 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:51.935461 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:51.951522 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:51.951490 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:17:51.963572 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:51.963525 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" podStartSLOduration=10.241431644 podStartE2EDuration="28.963507802s" totalImagesPullingTime="18.722076158s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:17:51.961897408 +0000 UTC m=+28.861166516" watchObservedRunningTime="2026-07-10 18:17:51.963507802 +0000 UTC m=+28.862776915" Jul 10 18:17:52.677664 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:52.677605 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:52.677958 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:52.677771 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsqx" podUID="29ea8cc9-50a4-4990-a85c-b8c47aa81f0b" Jul 10 18:17:52.677958 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:52.677817 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:52.677958 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:52.677899 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lftkj" podUID="0a36554d-7c38-4893-9dca-ceac16436f2e" Jul 10 18:17:52.777195 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:52.777149 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lftkj"] Jul 10 18:17:52.777369 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:52.777277 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9rqql"] Jul 10 18:17:52.777422 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:52.777391 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:52.777571 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:52.777543 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9rqql" podUID="91c02392-14bc-403d-ab9f-131ed9213d28" Jul 10 18:17:52.779489 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:52.779453 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ndsqx"] Jul 10 18:17:52.939678 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:52.939578 2571 generic.go:356] "Generic (PLEG): container finished" podID="6058e5b7-e082-4aff-a0b2-1a46422bdb3a" containerID="5e62c448be2d78d817fc30c0d4d4f9370915facfee3c7e6cd5adae7da5110a14" exitCode=0 Jul 10 18:17:52.939678 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:52.939650 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzc82" event={"ID":"6058e5b7-e082-4aff-a0b2-1a46422bdb3a","Type":"ContainerDied","Data":"5e62c448be2d78d817fc30c0d4d4f9370915facfee3c7e6cd5adae7da5110a14"} Jul 10 18:17:52.940104 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:52.939732 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:52.940104 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:52.939823 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsqx" podUID="29ea8cc9-50a4-4990-a85c-b8c47aa81f0b" Jul 10 18:17:52.940177 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:52.940156 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:52.940313 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:52.940289 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lftkj" podUID="0a36554d-7c38-4893-9dca-ceac16436f2e" Jul 10 18:17:54.677410 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:54.677369 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:54.678187 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:54.677369 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:54.678187 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:54.677512 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsqx" podUID="29ea8cc9-50a4-4990-a85c-b8c47aa81f0b" Jul 10 18:17:54.678187 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:54.677369 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:54.678187 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:54.677563 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9rqql" podUID="91c02392-14bc-403d-ab9f-131ed9213d28" Jul 10 18:17:54.678187 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:54.677645 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lftkj" podUID="0a36554d-7c38-4893-9dca-ceac16436f2e" Jul 10 18:17:54.949597 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:54.949509 2571 generic.go:356] "Generic (PLEG): container finished" podID="6058e5b7-e082-4aff-a0b2-1a46422bdb3a" containerID="4d642e473481c06c59f7119082d59c481e50700ccfa36529ded6d06e2c05670f" exitCode=0 Jul 10 18:17:54.949597 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:54.949562 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzc82" event={"ID":"6058e5b7-e082-4aff-a0b2-1a46422bdb3a","Type":"ContainerDied","Data":"4d642e473481c06c59f7119082d59c481e50700ccfa36529ded6d06e2c05670f"} Jul 10 18:17:56.678412 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:56.678120 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:56.678974 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:56.678158 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:56.678974 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:56.678516 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9rqql" podUID="91c02392-14bc-403d-ab9f-131ed9213d28" Jul 10 18:17:56.678974 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:56.678159 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:56.678974 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:56.678597 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lftkj" podUID="0a36554d-7c38-4893-9dca-ceac16436f2e" Jul 10 18:17:56.678974 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:56.678683 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsqx" podUID="29ea8cc9-50a4-4990-a85c-b8c47aa81f0b" Jul 10 18:17:57.241243 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:57.241202 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret\") pod \"global-pull-secret-syncer-9rqql\" (UID: \"91c02392-14bc-403d-ab9f-131ed9213d28\") " pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:57.241421 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:57.241347 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:57.241421 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:57.241415 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret podName:91c02392-14bc-403d-ab9f-131ed9213d28 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:29.241395643 +0000 UTC m=+66.140664733 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret") pod "global-pull-secret-syncer-9rqql" (UID: "91c02392-14bc-403d-ab9f-131ed9213d28") : object "kube-system"/"original-pull-secret" not registered Jul 10 18:17:57.342012 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:57.341966 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs\") pod \"network-metrics-daemon-ndsqx\" (UID: \"29ea8cc9-50a4-4990-a85c-b8c47aa81f0b\") " pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:57.342180 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:57.342118 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:57.342219 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:57.342181 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs podName:29ea8cc9-50a4-4990-a85c-b8c47aa81f0b nodeName:}" failed. No retries permitted until 2026-07-10 18:18:29.342167234 +0000 UTC m=+66.241436319 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs") pod "network-metrics-daemon-ndsqx" (UID: "29ea8cc9-50a4-4990-a85c-b8c47aa81f0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 18:17:57.442440 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:57.442409 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrmqx\" (UniqueName: \"kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx\") pod \"network-check-target-lftkj\" (UID: \"0a36554d-7c38-4893-9dca-ceac16436f2e\") " pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:57.442605 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:57.442578 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 18:17:57.442605 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:57.442600 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 18:17:57.442701 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:57.442623 2571 projected.go:196] Error preparing data for projected volume kube-api-access-wrmqx for pod openshift-network-diagnostics/network-check-target-lftkj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:57.442701 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:57.442679 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx podName:0a36554d-7c38-4893-9dca-ceac16436f2e nodeName:}" failed. No retries permitted until 2026-07-10 18:18:29.442664431 +0000 UTC m=+66.341933520 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wrmqx" (UniqueName: "kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx") pod "network-check-target-lftkj" (UID: "0a36554d-7c38-4893-9dca-ceac16436f2e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 18:17:58.402964 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.402928 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-54.ec2.internal" event="NodeReady" Jul 10 18:17:58.403442 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.403119 2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 18:17:58.451315 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.451274 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-cwtqn"] Jul 10 18:17:58.484107 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.484073 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xksd4"] Jul 10 18:17:58.484295 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.484222 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cwtqn" Jul 10 18:17:58.486938 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.486909 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 18:17:58.487080 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.486950 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lzjfw\"" Jul 10 18:17:58.487439 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.487199 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 18:17:58.487439 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.487274 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 18:17:58.499699 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.499443 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cwtqn"] Jul 10 18:17:58.499699 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.499485 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xksd4"] Jul 10 18:17:58.499699 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.499584 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xksd4" Jul 10 18:17:58.502020 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.501997 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 18:17:58.502153 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.502038 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 18:17:58.502153 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.502140 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p5qqq\"" Jul 10 18:17:58.651039 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.651000 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f17d7a9-1710-4246-9196-6c85ce329679-metrics-tls\") pod \"dns-default-xksd4\" (UID: \"2f17d7a9-1710-4246-9196-6c85ce329679\") " pod="openshift-dns/dns-default-xksd4" Jul 10 18:17:58.651200 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.651047 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f17d7a9-1710-4246-9196-6c85ce329679-tmp-dir\") pod \"dns-default-xksd4\" (UID: \"2f17d7a9-1710-4246-9196-6c85ce329679\") " pod="openshift-dns/dns-default-xksd4" Jul 10 18:17:58.651200 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.651087 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-956qn\" (UniqueName: \"kubernetes.io/projected/2f17d7a9-1710-4246-9196-6c85ce329679-kube-api-access-956qn\") pod \"dns-default-xksd4\" (UID: \"2f17d7a9-1710-4246-9196-6c85ce329679\") " pod="openshift-dns/dns-default-xksd4" Jul 10 18:17:58.651200 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.651143 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95b3a951-0550-4589-afc1-3255ff58e1ec-cert\") pod \"ingress-canary-cwtqn\" (UID: \"95b3a951-0550-4589-afc1-3255ff58e1ec\") " pod="openshift-ingress-canary/ingress-canary-cwtqn" Jul 10 18:17:58.651358 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.651229 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnfff\" (UniqueName: \"kubernetes.io/projected/95b3a951-0550-4589-afc1-3255ff58e1ec-kube-api-access-jnfff\") pod \"ingress-canary-cwtqn\" (UID: \"95b3a951-0550-4589-afc1-3255ff58e1ec\") " pod="openshift-ingress-canary/ingress-canary-cwtqn" Jul 10 18:17:58.651358 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.651281 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f17d7a9-1710-4246-9196-6c85ce329679-config-volume\") pod \"dns-default-xksd4\" (UID: \"2f17d7a9-1710-4246-9196-6c85ce329679\") " pod="openshift-dns/dns-default-xksd4" Jul 10 18:17:58.677902 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.677823 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:17:58.677902 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.677879 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:17:58.678111 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.678008 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:17:58.680784 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.680758 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nsh8q\"" Jul 10 18:17:58.680784 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.680758 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 18:17:58.680784 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.680785 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 18:17:58.681200 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.681180 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 18:17:58.681322 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.681206 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xs98z\"" Jul 10 18:17:58.681961 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.681937 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 18:17:58.752118 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.752076 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f17d7a9-1710-4246-9196-6c85ce329679-tmp-dir\") pod \"dns-default-xksd4\" (UID: \"2f17d7a9-1710-4246-9196-6c85ce329679\") " pod="openshift-dns/dns-default-xksd4" Jul 10 18:17:58.752118 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.752118 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-956qn\" (UniqueName: \"kubernetes.io/projected/2f17d7a9-1710-4246-9196-6c85ce329679-kube-api-access-956qn\") pod \"dns-default-xksd4\" (UID: \"2f17d7a9-1710-4246-9196-6c85ce329679\") " pod="openshift-dns/dns-default-xksd4" Jul 10 18:17:58.752379 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.752136 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95b3a951-0550-4589-afc1-3255ff58e1ec-cert\") pod \"ingress-canary-cwtqn\" (UID: \"95b3a951-0550-4589-afc1-3255ff58e1ec\") " pod="openshift-ingress-canary/ingress-canary-cwtqn" Jul 10 18:17:58.752379 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.752212 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnfff\" (UniqueName: \"kubernetes.io/projected/95b3a951-0550-4589-afc1-3255ff58e1ec-kube-api-access-jnfff\") pod \"ingress-canary-cwtqn\" (UID: \"95b3a951-0550-4589-afc1-3255ff58e1ec\") " pod="openshift-ingress-canary/ingress-canary-cwtqn" Jul 10 18:17:58.752379 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.752265 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f17d7a9-1710-4246-9196-6c85ce329679-config-volume\") pod \"dns-default-xksd4\" (UID: \"2f17d7a9-1710-4246-9196-6c85ce329679\") " pod="openshift-dns/dns-default-xksd4" Jul 10 18:17:58.752531 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.752447 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f17d7a9-1710-4246-9196-6c85ce329679-metrics-tls\") pod \"dns-default-xksd4\" (UID: \"2f17d7a9-1710-4246-9196-6c85ce329679\") " pod="openshift-dns/dns-default-xksd4" Jul 10 18:17:58.752531 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.752451 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f17d7a9-1710-4246-9196-6c85ce329679-tmp-dir\") pod \"dns-default-xksd4\" (UID: \"2f17d7a9-1710-4246-9196-6c85ce329679\") " pod="openshift-dns/dns-default-xksd4" Jul 10 18:17:58.752659 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:58.752529 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:17:58.752659 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:58.752595 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f17d7a9-1710-4246-9196-6c85ce329679-metrics-tls podName:2f17d7a9-1710-4246-9196-6c85ce329679 nodeName:}" failed. No retries permitted until 2026-07-10 18:17:59.252574707 +0000 UTC m=+36.151843812 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f17d7a9-1710-4246-9196-6c85ce329679-metrics-tls") pod "dns-default-xksd4" (UID: "2f17d7a9-1710-4246-9196-6c85ce329679") : secret "dns-default-metrics-tls" not found Jul 10 18:17:58.752885 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.752866 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f17d7a9-1710-4246-9196-6c85ce329679-config-volume\") pod \"dns-default-xksd4\" (UID: \"2f17d7a9-1710-4246-9196-6c85ce329679\") " pod="openshift-dns/dns-default-xksd4" Jul 10 18:17:58.757273 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.757248 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95b3a951-0550-4589-afc1-3255ff58e1ec-cert\") pod \"ingress-canary-cwtqn\" (UID: \"95b3a951-0550-4589-afc1-3255ff58e1ec\") " pod="openshift-ingress-canary/ingress-canary-cwtqn" Jul 10 18:17:58.761314 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.761289 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-956qn\" (UniqueName: \"kubernetes.io/projected/2f17d7a9-1710-4246-9196-6c85ce329679-kube-api-access-956qn\") pod \"dns-default-xksd4\" (UID: \"2f17d7a9-1710-4246-9196-6c85ce329679\") " pod="openshift-dns/dns-default-xksd4" Jul 10 18:17:58.761470 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.761373 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnfff\" (UniqueName: \"kubernetes.io/projected/95b3a951-0550-4589-afc1-3255ff58e1ec-kube-api-access-jnfff\") pod \"ingress-canary-cwtqn\" (UID: \"95b3a951-0550-4589-afc1-3255ff58e1ec\") " pod="openshift-ingress-canary/ingress-canary-cwtqn" Jul 10 18:17:58.795835 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:58.795798 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cwtqn" Jul 10 18:17:59.144201 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:59.144169 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cwtqn"] Jul 10 18:17:59.214159 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:17:59.214123 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b3a951_0550_4589_afc1_3255ff58e1ec.slice/crio-7b97f12550917fc74527862a9b753927bdf6eff0b537a56d50a63dba428f1b1c WatchSource:0}: Error finding container 7b97f12550917fc74527862a9b753927bdf6eff0b537a56d50a63dba428f1b1c: Status 404 returned error can't find the container with id 7b97f12550917fc74527862a9b753927bdf6eff0b537a56d50a63dba428f1b1c Jul 10 18:17:59.257267 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:59.257242 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f17d7a9-1710-4246-9196-6c85ce329679-metrics-tls\") pod \"dns-default-xksd4\" (UID: \"2f17d7a9-1710-4246-9196-6c85ce329679\") " pod="openshift-dns/dns-default-xksd4" Jul 10 18:17:59.257939 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:59.257420 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:17:59.257939 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:17:59.257510 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f17d7a9-1710-4246-9196-6c85ce329679-metrics-tls podName:2f17d7a9-1710-4246-9196-6c85ce329679 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:00.25748851 +0000 UTC m=+37.156757613 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f17d7a9-1710-4246-9196-6c85ce329679-metrics-tls") pod "dns-default-xksd4" (UID: "2f17d7a9-1710-4246-9196-6c85ce329679") : secret "dns-default-metrics-tls" not found Jul 10 18:17:59.965050 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:59.964859 2571 generic.go:356] "Generic (PLEG): container finished" podID="6058e5b7-e082-4aff-a0b2-1a46422bdb3a" containerID="d203277e4b8ec7a06865ecf10934d4e44566ec891977f06e3f83c965f6d95df2" exitCode=0 Jul 10 18:17:59.965514 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:59.964951 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzc82" event={"ID":"6058e5b7-e082-4aff-a0b2-1a46422bdb3a","Type":"ContainerDied","Data":"d203277e4b8ec7a06865ecf10934d4e44566ec891977f06e3f83c965f6d95df2"} Jul 10 18:17:59.966485 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:17:59.966460 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cwtqn" event={"ID":"95b3a951-0550-4589-afc1-3255ff58e1ec","Type":"ContainerStarted","Data":"7b97f12550917fc74527862a9b753927bdf6eff0b537a56d50a63dba428f1b1c"} Jul 10 18:18:00.265633 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:00.265582 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f17d7a9-1710-4246-9196-6c85ce329679-metrics-tls\") pod \"dns-default-xksd4\" (UID: \"2f17d7a9-1710-4246-9196-6c85ce329679\") " pod="openshift-dns/dns-default-xksd4" Jul 10 18:18:00.265831 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:00.265757 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:18:00.265891 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:00.265834 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f17d7a9-1710-4246-9196-6c85ce329679-metrics-tls podName:2f17d7a9-1710-4246-9196-6c85ce329679 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:02.26581324 +0000 UTC m=+39.165082341 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f17d7a9-1710-4246-9196-6c85ce329679-metrics-tls") pod "dns-default-xksd4" (UID: "2f17d7a9-1710-4246-9196-6c85ce329679") : secret "dns-default-metrics-tls" not found Jul 10 18:18:00.972710 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:00.972661 2571 generic.go:356] "Generic (PLEG): container finished" podID="6058e5b7-e082-4aff-a0b2-1a46422bdb3a" containerID="0aa44fece6bcfb1b759238fa497101e9e445879215cb4d9f2172a0951760e4a9" exitCode=0 Jul 10 18:18:00.973183 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:00.972736 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzc82" event={"ID":"6058e5b7-e082-4aff-a0b2-1a46422bdb3a","Type":"ContainerDied","Data":"0aa44fece6bcfb1b759238fa497101e9e445879215cb4d9f2172a0951760e4a9"} Jul 10 18:18:01.976918 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:01.976813 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cwtqn" event={"ID":"95b3a951-0550-4589-afc1-3255ff58e1ec","Type":"ContainerStarted","Data":"ce69ac6f461a9777e881ead1a96372adf2e57d7b5358346c27af9f1192301d52"} Jul 10 18:18:01.979991 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:01.979965 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzc82" event={"ID":"6058e5b7-e082-4aff-a0b2-1a46422bdb3a","Type":"ContainerStarted","Data":"229432c550243b34ba3f24ec450d5c2d3f1a28d634a5f37743bca1b0b2d2e1f4"} Jul 10 18:18:01.992829 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:01.992783 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-cwtqn" podStartSLOduration=1.581443412 podStartE2EDuration="3.992766922s" totalImagesPullingTime="2.41132351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:17:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:01.991409875 +0000 UTC m=+38.890678987" watchObservedRunningTime="2026-07-10 18:18:01.992766922 +0000 UTC m=+38.892036029" Jul 10 18:18:02.014211 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:02.014006 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fzc82" podStartSLOduration=9.380314731 podStartE2EDuration="39.013985395s" totalImagesPullingTime="29.633670664s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:02.012291826 +0000 UTC m=+38.911560934" watchObservedRunningTime="2026-07-10 18:18:02.013985395 +0000 UTC m=+38.913254504" Jul 10 18:18:02.281287 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:02.281255 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f17d7a9-1710-4246-9196-6c85ce329679-metrics-tls\") pod \"dns-default-xksd4\" (UID: \"2f17d7a9-1710-4246-9196-6c85ce329679\") " pod="openshift-dns/dns-default-xksd4" Jul 10 18:18:02.281448 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:02.281382 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:18:02.281485 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:02.281448 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f17d7a9-1710-4246-9196-6c85ce329679-metrics-tls podName:2f17d7a9-1710-4246-9196-6c85ce329679 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:06.281435748 +0000 UTC m=+43.180704834 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f17d7a9-1710-4246-9196-6c85ce329679-metrics-tls") pod "dns-default-xksd4" (UID: "2f17d7a9-1710-4246-9196-6c85ce329679") : secret "dns-default-metrics-tls" not found Jul 10 18:18:05.383372 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:05.383338 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5hhgq"] Jul 10 18:18:05.398351 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:05.398314 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5hhgq"] Jul 10 18:18:05.398517 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:05.398454 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:05.402817 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:05.402790 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 18:18:05.403313 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:05.403255 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 18:18:05.404062 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:05.404036 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qtcxd\"" Jul 10 18:18:05.404062 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:05.404057 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 18:18:05.404240 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:05.404079 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 18:18:05.502064 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:05.502031 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/92eb18e4-693d-4dd4-8948-501fbc8571eb-data-volume\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:05.502064 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:05.502066 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24sj5\" (UniqueName: \"kubernetes.io/projected/92eb18e4-693d-4dd4-8948-501fbc8571eb-kube-api-access-24sj5\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:05.502277 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:05.502097 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92eb18e4-693d-4dd4-8948-501fbc8571eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:05.502277 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:05.502179 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92eb18e4-693d-4dd4-8948-501fbc8571eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:05.502277 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:05.502238 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/92eb18e4-693d-4dd4-8948-501fbc8571eb-crio-socket\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:05.603095 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:05.603059 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/92eb18e4-693d-4dd4-8948-501fbc8571eb-data-volume\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:05.603095 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:05.603100 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24sj5\" (UniqueName: \"kubernetes.io/projected/92eb18e4-693d-4dd4-8948-501fbc8571eb-kube-api-access-24sj5\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:05.603310 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:05.603125 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92eb18e4-693d-4dd4-8948-501fbc8571eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:05.603310 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:05.603149 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92eb18e4-693d-4dd4-8948-501fbc8571eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:05.603310 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:05.603188 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/92eb18e4-693d-4dd4-8948-501fbc8571eb-crio-socket\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:05.603310 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:05.603290 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:18:05.603492 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:05.603357 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/92eb18e4-693d-4dd4-8948-501fbc8571eb-crio-socket\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:05.603492 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:05.603369 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/92eb18e4-693d-4dd4-8948-501fbc8571eb-kube-rbac-proxy-cm podName:92eb18e4-693d-4dd4-8948-501fbc8571eb nodeName:}" failed. No retries permitted until 2026-07-10 18:18:06.103348119 +0000 UTC m=+43.002617222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/92eb18e4-693d-4dd4-8948-501fbc8571eb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5hhgq" (UID: "92eb18e4-693d-4dd4-8948-501fbc8571eb") : configmap "kube-rbac-proxy" not found Jul 10 18:18:05.603492 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:05.603295 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:18:05.603492 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:05.603381 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/92eb18e4-693d-4dd4-8948-501fbc8571eb-data-volume\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:05.603492 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:05.603430 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92eb18e4-693d-4dd4-8948-501fbc8571eb-insights-runtime-extractor-tls podName:92eb18e4-693d-4dd4-8948-501fbc8571eb nodeName:}" failed. No retries permitted until 2026-07-10 18:18:06.103412864 +0000 UTC m=+43.002681962 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/92eb18e4-693d-4dd4-8948-501fbc8571eb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5hhgq" (UID: "92eb18e4-693d-4dd4-8948-501fbc8571eb") : secret "insights-runtime-extractor-tls" not found Jul 10 18:18:05.614608 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:05.614582 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24sj5\" (UniqueName: \"kubernetes.io/projected/92eb18e4-693d-4dd4-8948-501fbc8571eb-kube-api-access-24sj5\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:06.106987 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:06.106940 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92eb18e4-693d-4dd4-8948-501fbc8571eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:06.107148 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:06.107056 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92eb18e4-693d-4dd4-8948-501fbc8571eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:06.107148 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:06.107073 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:18:06.107148 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:06.107137 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/92eb18e4-693d-4dd4-8948-501fbc8571eb-kube-rbac-proxy-cm podName:92eb18e4-693d-4dd4-8948-501fbc8571eb nodeName:}" failed. No retries permitted until 2026-07-10 18:18:07.107120149 +0000 UTC m=+44.006389236 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/92eb18e4-693d-4dd4-8948-501fbc8571eb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5hhgq" (UID: "92eb18e4-693d-4dd4-8948-501fbc8571eb") : configmap "kube-rbac-proxy" not found Jul 10 18:18:06.107288 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:06.107166 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:18:06.107288 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:06.107213 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92eb18e4-693d-4dd4-8948-501fbc8571eb-insights-runtime-extractor-tls podName:92eb18e4-693d-4dd4-8948-501fbc8571eb nodeName:}" failed. No retries permitted until 2026-07-10 18:18:07.107199409 +0000 UTC m=+44.006468500 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/92eb18e4-693d-4dd4-8948-501fbc8571eb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5hhgq" (UID: "92eb18e4-693d-4dd4-8948-501fbc8571eb") : secret "insights-runtime-extractor-tls" not found Jul 10 18:18:06.307808 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:06.307767 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f17d7a9-1710-4246-9196-6c85ce329679-metrics-tls\") pod \"dns-default-xksd4\" (UID: \"2f17d7a9-1710-4246-9196-6c85ce329679\") " pod="openshift-dns/dns-default-xksd4" Jul 10 18:18:06.307987 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:06.307902 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:18:06.307987 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:06.307956 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f17d7a9-1710-4246-9196-6c85ce329679-metrics-tls podName:2f17d7a9-1710-4246-9196-6c85ce329679 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:14.307940627 +0000 UTC m=+51.207209712 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f17d7a9-1710-4246-9196-6c85ce329679-metrics-tls") pod "dns-default-xksd4" (UID: "2f17d7a9-1710-4246-9196-6c85ce329679") : secret "dns-default-metrics-tls" not found Jul 10 18:18:07.113730 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:07.113695 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92eb18e4-693d-4dd4-8948-501fbc8571eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:07.114128 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:07.113742 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92eb18e4-693d-4dd4-8948-501fbc8571eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:07.114128 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:07.113823 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:18:07.114128 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:07.113863 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:18:07.114128 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:07.113874 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/92eb18e4-693d-4dd4-8948-501fbc8571eb-kube-rbac-proxy-cm podName:92eb18e4-693d-4dd4-8948-501fbc8571eb nodeName:}" failed. No retries permitted until 2026-07-10 18:18:09.113859442 +0000 UTC m=+46.013128528 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/92eb18e4-693d-4dd4-8948-501fbc8571eb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5hhgq" (UID: "92eb18e4-693d-4dd4-8948-501fbc8571eb") : configmap "kube-rbac-proxy" not found Jul 10 18:18:07.114128 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:07.113937 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92eb18e4-693d-4dd4-8948-501fbc8571eb-insights-runtime-extractor-tls podName:92eb18e4-693d-4dd4-8948-501fbc8571eb nodeName:}" failed. No retries permitted until 2026-07-10 18:18:09.113924684 +0000 UTC m=+46.013193771 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/92eb18e4-693d-4dd4-8948-501fbc8571eb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5hhgq" (UID: "92eb18e4-693d-4dd4-8948-501fbc8571eb") : secret "insights-runtime-extractor-tls" not found Jul 10 18:18:07.357826 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:07.357800 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kt69c_05e2da1a-b49d-47e5-85e8-2ce1d30d3aae/dns-node-resolver/0.log" Jul 10 18:18:07.890900 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:07.890865 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-v6xpt"] Jul 10 18:18:07.919524 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:07.919482 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-v6xpt"] Jul 10 18:18:07.919718 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:07.919596 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-v6xpt" Jul 10 18:18:07.922735 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:07.922702 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 10 18:18:07.922934 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:07.922921 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 10 18:18:07.924064 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:07.924047 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-fs6vb\"" Jul 10 18:18:08.020976 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:08.020933 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggtnv\" (UniqueName: \"kubernetes.io/projected/28465aca-96df-43d6-8c90-42a88fb716ca-kube-api-access-ggtnv\") pod \"migrator-7f59b54f6c-v6xpt\" (UID: \"28465aca-96df-43d6-8c90-42a88fb716ca\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-v6xpt" Jul 10 18:18:08.122010 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:08.121971 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggtnv\" (UniqueName: \"kubernetes.io/projected/28465aca-96df-43d6-8c90-42a88fb716ca-kube-api-access-ggtnv\") pod \"migrator-7f59b54f6c-v6xpt\" (UID: \"28465aca-96df-43d6-8c90-42a88fb716ca\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-v6xpt" Jul 10 18:18:08.130795 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:08.130766 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggtnv\" (UniqueName: \"kubernetes.io/projected/28465aca-96df-43d6-8c90-42a88fb716ca-kube-api-access-ggtnv\") pod \"migrator-7f59b54f6c-v6xpt\" (UID: \"28465aca-96df-43d6-8c90-42a88fb716ca\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-v6xpt" Jul 10 18:18:08.157557 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:08.157481 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6jpkg_a3929aa3-2abd-489c-b7fd-c80e982db8e8/node-ca/0.log" Jul 10 18:18:08.228783 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:08.228744 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-v6xpt" Jul 10 18:18:08.375184 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:08.375150 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-v6xpt"] Jul 10 18:18:08.378372 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:18:08.378344 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28465aca_96df_43d6_8c90_42a88fb716ca.slice/crio-9bdb2a8cdeab8fe19557fec555e4b85d639499752ce8695a8569b9463da01844 WatchSource:0}: Error finding container 9bdb2a8cdeab8fe19557fec555e4b85d639499752ce8695a8569b9463da01844: Status 404 returned error can't find the container with id 9bdb2a8cdeab8fe19557fec555e4b85d639499752ce8695a8569b9463da01844 Jul 10 18:18:08.998768 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:08.998733 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-v6xpt" event={"ID":"28465aca-96df-43d6-8c90-42a88fb716ca","Type":"ContainerStarted","Data":"9bdb2a8cdeab8fe19557fec555e4b85d639499752ce8695a8569b9463da01844"} Jul 10 18:18:09.129153 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:09.129106 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92eb18e4-693d-4dd4-8948-501fbc8571eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:09.129663 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:09.129163 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92eb18e4-693d-4dd4-8948-501fbc8571eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:09.129663 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:09.129276 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:18:09.129663 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:09.129300 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 18:18:09.129663 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:09.129360 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92eb18e4-693d-4dd4-8948-501fbc8571eb-insights-runtime-extractor-tls podName:92eb18e4-693d-4dd4-8948-501fbc8571eb nodeName:}" failed. No retries permitted until 2026-07-10 18:18:13.129338517 +0000 UTC m=+50.028607606 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/92eb18e4-693d-4dd4-8948-501fbc8571eb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5hhgq" (UID: "92eb18e4-693d-4dd4-8948-501fbc8571eb") : secret "insights-runtime-extractor-tls" not found Jul 10 18:18:09.129663 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:09.129381 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/92eb18e4-693d-4dd4-8948-501fbc8571eb-kube-rbac-proxy-cm podName:92eb18e4-693d-4dd4-8948-501fbc8571eb nodeName:}" failed. No retries permitted until 2026-07-10 18:18:13.129370699 +0000 UTC m=+50.028639786 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/92eb18e4-693d-4dd4-8948-501fbc8571eb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5hhgq" (UID: "92eb18e4-693d-4dd4-8948-501fbc8571eb") : configmap "kube-rbac-proxy" not found Jul 10 18:18:09.168338 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:09.168305 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cwtqn_95b3a951-0550-4589-afc1-3255ff58e1ec/serve-healthcheck-canary/0.log" Jul 10 18:18:11.005934 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:11.005895 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-v6xpt" event={"ID":"28465aca-96df-43d6-8c90-42a88fb716ca","Type":"ContainerStarted","Data":"b0522062116c530bc2818aa787577b479293bc579da576d533ea0f7f3261ab89"} Jul 10 18:18:11.006401 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:11.005956 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-v6xpt" event={"ID":"28465aca-96df-43d6-8c90-42a88fb716ca","Type":"ContainerStarted","Data":"eec23d6fca83d4a3a062906c168fc2a8dee8d5505afa54eeec14e8cc960c22c0"} Jul 10 18:18:13.163178 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:13.163133 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92eb18e4-693d-4dd4-8948-501fbc8571eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:13.163178 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:13.163185 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92eb18e4-693d-4dd4-8948-501fbc8571eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:13.163608 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:13.163289 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 18:18:13.163608 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:13.163366 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92eb18e4-693d-4dd4-8948-501fbc8571eb-insights-runtime-extractor-tls podName:92eb18e4-693d-4dd4-8948-501fbc8571eb nodeName:}" failed. No retries permitted until 2026-07-10 18:18:21.163350815 +0000 UTC m=+58.062619900 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/92eb18e4-693d-4dd4-8948-501fbc8571eb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5hhgq" (UID: "92eb18e4-693d-4dd4-8948-501fbc8571eb") : secret "insights-runtime-extractor-tls" not found Jul 10 18:18:13.163810 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:13.163792 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92eb18e4-693d-4dd4-8948-501fbc8571eb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:14.373400 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:14.373358 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f17d7a9-1710-4246-9196-6c85ce329679-metrics-tls\") pod \"dns-default-xksd4\" (UID: \"2f17d7a9-1710-4246-9196-6c85ce329679\") " pod="openshift-dns/dns-default-xksd4" Jul 10 18:18:14.373811 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:14.373488 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 18:18:14.373811 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:18:14.373542 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f17d7a9-1710-4246-9196-6c85ce329679-metrics-tls podName:2f17d7a9-1710-4246-9196-6c85ce329679 nodeName:}" failed. No retries permitted until 2026-07-10 18:18:30.373527922 +0000 UTC m=+67.272797007 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f17d7a9-1710-4246-9196-6c85ce329679-metrics-tls") pod "dns-default-xksd4" (UID: "2f17d7a9-1710-4246-9196-6c85ce329679") : secret "dns-default-metrics-tls" not found Jul 10 18:18:21.219234 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:21.219190 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92eb18e4-693d-4dd4-8948-501fbc8571eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:21.221543 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:21.221516 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92eb18e4-693d-4dd4-8948-501fbc8571eb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5hhgq\" (UID: \"92eb18e4-693d-4dd4-8948-501fbc8571eb\") " pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:21.308419 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:21.308377 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5hhgq" Jul 10 18:18:21.461945 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:21.461892 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-v6xpt" podStartSLOduration=12.584468445 podStartE2EDuration="14.461873114s" totalImagesPullingTime="1.877404669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:11.022329111 +0000 UTC m=+47.921598238" watchObservedRunningTime="2026-07-10 18:18:21.461873114 +0000 UTC m=+58.361142221" Jul 10 18:18:21.462106 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:21.462093 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5hhgq"] Jul 10 18:18:22.036677 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:22.036626 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5hhgq" event={"ID":"92eb18e4-693d-4dd4-8948-501fbc8571eb","Type":"ContainerStarted","Data":"e8ab319481f1c9731ed121d15548e66c15124bb6ce21d33d27aadc26135f746e"} Jul 10 18:18:23.954950 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:23.954910 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7r9vz" Jul 10 18:18:26.049255 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:26.049218 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5hhgq" event={"ID":"92eb18e4-693d-4dd4-8948-501fbc8571eb","Type":"ContainerStarted","Data":"65ac45d2723914c8ec64fd9730bc0e9d5ce24b80157962170ae9aabb09d1fb54"} Jul 10 18:18:26.049797 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:26.049284 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5hhgq" event={"ID":"92eb18e4-693d-4dd4-8948-501fbc8571eb","Type":"ContainerStarted","Data":"5a4639f8a5bb71a003adee9e637d379ffc4c3e604445bddad129867646c586be"} Jul 10 18:18:29.060448 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.060415 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5hhgq" event={"ID":"92eb18e4-693d-4dd4-8948-501fbc8571eb","Type":"ContainerStarted","Data":"d814b072280f0820f9da7d8e7e20418b252b99a0966ff1aebdaddbffc2e38b64"} Jul 10 18:18:29.079077 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.079026 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5hhgq" podStartSLOduration=17.376280123 podStartE2EDuration="24.079010418s" totalImagesPullingTime="6.702730295s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:18:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:29.077493642 +0000 UTC m=+65.976762750" watchObservedRunningTime="2026-07-10 18:18:29.079010418 +0000 UTC m=+65.978279539" Jul 10 18:18:29.285632 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.285580 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret\") pod \"global-pull-secret-syncer-9rqql\" (UID: \"91c02392-14bc-403d-ab9f-131ed9213d28\") " pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:18:29.288334 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.288316 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 18:18:29.298646 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.298606 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/91c02392-14bc-403d-ab9f-131ed9213d28-original-pull-secret\") pod \"global-pull-secret-syncer-9rqql\" (UID: \"91c02392-14bc-403d-ab9f-131ed9213d28\") " pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:18:29.302570 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.302545 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9rqql" Jul 10 18:18:29.386169 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.386133 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs\") pod \"network-metrics-daemon-ndsqx\" (UID: \"29ea8cc9-50a4-4990-a85c-b8c47aa81f0b\") " pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:18:29.390279 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.390205 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d86479bb-kvth6"] Jul 10 18:18:29.391495 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.391475 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 18:18:29.393082 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.393057 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d86479bb-kvth6" Jul 10 18:18:29.399760 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.399736 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29ea8cc9-50a4-4990-a85c-b8c47aa81f0b-metrics-certs\") pod \"network-metrics-daemon-ndsqx\" (UID: \"29ea8cc9-50a4-4990-a85c-b8c47aa81f0b\") " pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:18:29.406594 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.406569 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 10 18:18:29.406594 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.406586 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 10 18:18:29.406802 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.406656 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 10 18:18:29.407383 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.407366 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 10 18:18:29.407475 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.407412 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-t4zgn\"" Jul 10 18:18:29.407753 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.407734 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn"] Jul 10 18:18:29.409662 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.409647 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" Jul 10 18:18:29.414950 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.414932 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 10 18:18:29.419375 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.419346 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 10 18:18:29.421040 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.421022 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 10 18:18:29.421237 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.421223 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 10 18:18:29.428516 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.428492 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d86479bb-kvth6"] Jul 10 18:18:29.441307 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.441279 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn"] Jul 10 18:18:29.453690 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.453660 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9rqql"] Jul 10 18:18:29.456541 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:18:29.456513 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91c02392_14bc_403d_ab9f_131ed9213d28.slice/crio-018d0965b178ec6c13037100233eab16422bb21ad39a50ed1c87ffa0085ac1f8 WatchSource:0}: Error finding container 018d0965b178ec6c13037100233eab16422bb21ad39a50ed1c87ffa0085ac1f8: Status 404 returned error can't find the container with id 018d0965b178ec6c13037100233eab16422bb21ad39a50ed1c87ffa0085ac1f8 Jul 10 18:18:29.487093 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.487054 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckdl4\" (UniqueName: \"kubernetes.io/projected/0bb8f3dc-1669-4a8c-8b1f-9486cb198294-kube-api-access-ckdl4\") pod \"managed-serviceaccount-addon-agent-57d86479bb-kvth6\" (UID: \"0bb8f3dc-1669-4a8c-8b1f-9486cb198294\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d86479bb-kvth6" Jul 10 18:18:29.487257 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.487100 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrmqx\" (UniqueName: \"kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx\") pod \"network-check-target-lftkj\" (UID: \"0a36554d-7c38-4893-9dca-ceac16436f2e\") " pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:18:29.487257 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.487143 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/614d7858-1ef3-43d2-92ee-b0479f86d8fc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d4548c964-p8crn\" (UID: \"614d7858-1ef3-43d2-92ee-b0479f86d8fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" Jul 10 18:18:29.487257 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.487178 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/614d7858-1ef3-43d2-92ee-b0479f86d8fc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d4548c964-p8crn\" (UID: \"614d7858-1ef3-43d2-92ee-b0479f86d8fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" Jul 10 18:18:29.487257 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.487242 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0bb8f3dc-1669-4a8c-8b1f-9486cb198294-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-57d86479bb-kvth6\" (UID: \"0bb8f3dc-1669-4a8c-8b1f-9486cb198294\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d86479bb-kvth6" Jul 10 18:18:29.487459 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.487267 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/614d7858-1ef3-43d2-92ee-b0479f86d8fc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d4548c964-p8crn\" (UID: \"614d7858-1ef3-43d2-92ee-b0479f86d8fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" Jul 10 18:18:29.487459 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.487333 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/614d7858-1ef3-43d2-92ee-b0479f86d8fc-hub\") pod \"cluster-proxy-proxy-agent-6d4548c964-p8crn\" (UID: \"614d7858-1ef3-43d2-92ee-b0479f86d8fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" Jul 10 18:18:29.487459 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.487372 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/614d7858-1ef3-43d2-92ee-b0479f86d8fc-ca\") pod \"cluster-proxy-proxy-agent-6d4548c964-p8crn\" (UID: \"614d7858-1ef3-43d2-92ee-b0479f86d8fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" Jul 10 18:18:29.487459 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.487404 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w2zs\" (UniqueName: \"kubernetes.io/projected/614d7858-1ef3-43d2-92ee-b0479f86d8fc-kube-api-access-9w2zs\") pod \"cluster-proxy-proxy-agent-6d4548c964-p8crn\" (UID: \"614d7858-1ef3-43d2-92ee-b0479f86d8fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" Jul 10 18:18:29.489986 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.489968 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 18:18:29.501472 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.501438 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 18:18:29.511375 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.511349 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrmqx\" (UniqueName: \"kubernetes.io/projected/0a36554d-7c38-4893-9dca-ceac16436f2e-kube-api-access-wrmqx\") pod \"network-check-target-lftkj\" (UID: \"0a36554d-7c38-4893-9dca-ceac16436f2e\") " pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:18:29.588343 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.588242 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0bb8f3dc-1669-4a8c-8b1f-9486cb198294-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-57d86479bb-kvth6\" (UID: \"0bb8f3dc-1669-4a8c-8b1f-9486cb198294\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d86479bb-kvth6" Jul 10 18:18:29.588343 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.588284 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/614d7858-1ef3-43d2-92ee-b0479f86d8fc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d4548c964-p8crn\" (UID: \"614d7858-1ef3-43d2-92ee-b0479f86d8fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" Jul 10 18:18:29.588343 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.588304 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/614d7858-1ef3-43d2-92ee-b0479f86d8fc-hub\") pod \"cluster-proxy-proxy-agent-6d4548c964-p8crn\" (UID: \"614d7858-1ef3-43d2-92ee-b0479f86d8fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" Jul 10 18:18:29.588645 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.588443 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/614d7858-1ef3-43d2-92ee-b0479f86d8fc-ca\") pod \"cluster-proxy-proxy-agent-6d4548c964-p8crn\" (UID: \"614d7858-1ef3-43d2-92ee-b0479f86d8fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" Jul 10 18:18:29.588645 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.588489 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w2zs\" (UniqueName: \"kubernetes.io/projected/614d7858-1ef3-43d2-92ee-b0479f86d8fc-kube-api-access-9w2zs\") pod \"cluster-proxy-proxy-agent-6d4548c964-p8crn\" (UID: \"614d7858-1ef3-43d2-92ee-b0479f86d8fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" Jul 10 18:18:29.588645 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.588522 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckdl4\" (UniqueName: \"kubernetes.io/projected/0bb8f3dc-1669-4a8c-8b1f-9486cb198294-kube-api-access-ckdl4\") pod \"managed-serviceaccount-addon-agent-57d86479bb-kvth6\" (UID: \"0bb8f3dc-1669-4a8c-8b1f-9486cb198294\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d86479bb-kvth6" Jul 10 18:18:29.588645 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.588580 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/614d7858-1ef3-43d2-92ee-b0479f86d8fc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d4548c964-p8crn\" (UID: \"614d7858-1ef3-43d2-92ee-b0479f86d8fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" Jul 10 18:18:29.588645 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.588644 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/614d7858-1ef3-43d2-92ee-b0479f86d8fc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d4548c964-p8crn\" (UID: \"614d7858-1ef3-43d2-92ee-b0479f86d8fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" Jul 10 18:18:29.589353 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.589325 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/614d7858-1ef3-43d2-92ee-b0479f86d8fc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d4548c964-p8crn\" (UID: \"614d7858-1ef3-43d2-92ee-b0479f86d8fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" Jul 10 18:18:29.590897 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.590874 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/614d7858-1ef3-43d2-92ee-b0479f86d8fc-ca\") pod \"cluster-proxy-proxy-agent-6d4548c964-p8crn\" (UID: \"614d7858-1ef3-43d2-92ee-b0479f86d8fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" Jul 10 18:18:29.591024 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.591005 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/614d7858-1ef3-43d2-92ee-b0479f86d8fc-hub\") pod \"cluster-proxy-proxy-agent-6d4548c964-p8crn\" (UID: \"614d7858-1ef3-43d2-92ee-b0479f86d8fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" Jul 10 18:18:29.591077 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.591027 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0bb8f3dc-1669-4a8c-8b1f-9486cb198294-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-57d86479bb-kvth6\" (UID: \"0bb8f3dc-1669-4a8c-8b1f-9486cb198294\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d86479bb-kvth6" Jul 10 18:18:29.591124 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.591106 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/614d7858-1ef3-43d2-92ee-b0479f86d8fc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d4548c964-p8crn\" (UID: \"614d7858-1ef3-43d2-92ee-b0479f86d8fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" Jul 10 18:18:29.591250 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.591236 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/614d7858-1ef3-43d2-92ee-b0479f86d8fc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d4548c964-p8crn\" (UID: \"614d7858-1ef3-43d2-92ee-b0479f86d8fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" Jul 10 18:18:29.591917 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.591904 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nsh8q\"" Jul 10 18:18:29.598078 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.598060 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckdl4\" (UniqueName: \"kubernetes.io/projected/0bb8f3dc-1669-4a8c-8b1f-9486cb198294-kube-api-access-ckdl4\") pod \"managed-serviceaccount-addon-agent-57d86479bb-kvth6\" (UID: \"0bb8f3dc-1669-4a8c-8b1f-9486cb198294\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d86479bb-kvth6" Jul 10 18:18:29.598184 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.598169 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w2zs\" (UniqueName: \"kubernetes.io/projected/614d7858-1ef3-43d2-92ee-b0479f86d8fc-kube-api-access-9w2zs\") pod \"cluster-proxy-proxy-agent-6d4548c964-p8crn\" (UID: \"614d7858-1ef3-43d2-92ee-b0479f86d8fc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" Jul 10 18:18:29.598575 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.598561 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xs98z\"" Jul 10 18:18:29.599603 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.599589 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:18:29.606946 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.606925 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsqx" Jul 10 18:18:29.716303 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.716274 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d86479bb-kvth6" Jul 10 18:18:29.722581 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.722561 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" Jul 10 18:18:29.728388 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.728357 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lftkj"] Jul 10 18:18:29.732586 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:18:29.732562 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a36554d_7c38_4893_9dca_ceac16436f2e.slice/crio-05d9f80f19c84e70106f731eb57ffd58cacdc4d85ec0b4e9cf836099e294c280 WatchSource:0}: Error finding container 05d9f80f19c84e70106f731eb57ffd58cacdc4d85ec0b4e9cf836099e294c280: Status 404 returned error can't find the container with id 05d9f80f19c84e70106f731eb57ffd58cacdc4d85ec0b4e9cf836099e294c280 Jul 10 18:18:29.755397 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.755296 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ndsqx"] Jul 10 18:18:29.759838 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:18:29.759805 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29ea8cc9_50a4_4990_a85c_b8c47aa81f0b.slice/crio-094cd24f44cc4197c23d21be4167e69f82eefb95ee9a9843ba3141c3b405b864 WatchSource:0}: Error finding container 094cd24f44cc4197c23d21be4167e69f82eefb95ee9a9843ba3141c3b405b864: Status 404 returned error can't find the container with id 094cd24f44cc4197c23d21be4167e69f82eefb95ee9a9843ba3141c3b405b864 Jul 10 18:18:29.854385 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.854360 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d86479bb-kvth6"] Jul 10 18:18:29.856872 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:18:29.856833 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0bb8f3dc_1669_4a8c_8b1f_9486cb198294.slice/crio-4039ffe9894a0a55acaf4df0857ef70cc568deefbe0a03d6df88404a42655071 WatchSource:0}: Error finding container 4039ffe9894a0a55acaf4df0857ef70cc568deefbe0a03d6df88404a42655071: Status 404 returned error can't find the container with id 4039ffe9894a0a55acaf4df0857ef70cc568deefbe0a03d6df88404a42655071 Jul 10 18:18:29.876068 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:29.876018 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn"] Jul 10 18:18:29.880334 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:18:29.880305 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod614d7858_1ef3_43d2_92ee_b0479f86d8fc.slice/crio-6f835ada6c623d343ac0d2da1f3bd57725dc2d68b9186c3e19a1cf7b013cdb84 WatchSource:0}: Error finding container 6f835ada6c623d343ac0d2da1f3bd57725dc2d68b9186c3e19a1cf7b013cdb84: Status 404 returned error can't find the container with id 6f835ada6c623d343ac0d2da1f3bd57725dc2d68b9186c3e19a1cf7b013cdb84 Jul 10 18:18:30.065094 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:30.065056 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d86479bb-kvth6" event={"ID":"0bb8f3dc-1669-4a8c-8b1f-9486cb198294","Type":"ContainerStarted","Data":"4039ffe9894a0a55acaf4df0857ef70cc568deefbe0a03d6df88404a42655071"} Jul 10 18:18:30.066682 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:30.066647 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lftkj" event={"ID":"0a36554d-7c38-4893-9dca-ceac16436f2e","Type":"ContainerStarted","Data":"05d9f80f19c84e70106f731eb57ffd58cacdc4d85ec0b4e9cf836099e294c280"} Jul 10 18:18:30.067888 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:30.067850 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" event={"ID":"614d7858-1ef3-43d2-92ee-b0479f86d8fc","Type":"ContainerStarted","Data":"6f835ada6c623d343ac0d2da1f3bd57725dc2d68b9186c3e19a1cf7b013cdb84"} Jul 10 18:18:30.069117 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:30.069080 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ndsqx" event={"ID":"29ea8cc9-50a4-4990-a85c-b8c47aa81f0b","Type":"ContainerStarted","Data":"094cd24f44cc4197c23d21be4167e69f82eefb95ee9a9843ba3141c3b405b864"} Jul 10 18:18:30.070187 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:30.070164 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9rqql" event={"ID":"91c02392-14bc-403d-ab9f-131ed9213d28","Type":"ContainerStarted","Data":"018d0965b178ec6c13037100233eab16422bb21ad39a50ed1c87ffa0085ac1f8"} Jul 10 18:18:30.394828 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:30.394787 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f17d7a9-1710-4246-9196-6c85ce329679-metrics-tls\") pod \"dns-default-xksd4\" (UID: \"2f17d7a9-1710-4246-9196-6c85ce329679\") " pod="openshift-dns/dns-default-xksd4" Jul 10 18:18:30.397700 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:30.397640 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f17d7a9-1710-4246-9196-6c85ce329679-metrics-tls\") pod \"dns-default-xksd4\" (UID: \"2f17d7a9-1710-4246-9196-6c85ce329679\") " pod="openshift-dns/dns-default-xksd4" Jul 10 18:18:30.613412 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:30.613368 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p5qqq\"" Jul 10 18:18:30.621156 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:30.621124 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xksd4" Jul 10 18:18:31.237784 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:31.237703 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xksd4"] Jul 10 18:18:31.247238 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:18:31.247198 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f17d7a9_1710_4246_9196_6c85ce329679.slice/crio-e682b663cf15b37a1000ece83ec5239d60608164792f0b51f1ad16c438b49f40 WatchSource:0}: Error finding container e682b663cf15b37a1000ece83ec5239d60608164792f0b51f1ad16c438b49f40: Status 404 returned error can't find the container with id e682b663cf15b37a1000ece83ec5239d60608164792f0b51f1ad16c438b49f40 Jul 10 18:18:32.106512 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:32.106471 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xksd4" event={"ID":"2f17d7a9-1710-4246-9196-6c85ce329679","Type":"ContainerStarted","Data":"e682b663cf15b37a1000ece83ec5239d60608164792f0b51f1ad16c438b49f40"} Jul 10 18:18:32.117664 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:32.117197 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ndsqx" event={"ID":"29ea8cc9-50a4-4990-a85c-b8c47aa81f0b","Type":"ContainerStarted","Data":"e7d06e241454f69c80b9756aa0d44cb0253f6610048be5f1020d27a8295b1c41"} Jul 10 18:18:32.117664 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:32.117264 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ndsqx" event={"ID":"29ea8cc9-50a4-4990-a85c-b8c47aa81f0b","Type":"ContainerStarted","Data":"2293dabf10a65dea7f412599997aa8da2253af5e4dd247f35f2ed6b0ce4036e3"} Jul 10 18:18:39.146289 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:39.146244 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" event={"ID":"614d7858-1ef3-43d2-92ee-b0479f86d8fc","Type":"ContainerStarted","Data":"7de50139deeb151acc3b4003cc312fa745257f2614bd8bc109932b5e303ab3ba"} Jul 10 18:18:39.146737 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:39.146299 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" event={"ID":"614d7858-1ef3-43d2-92ee-b0479f86d8fc","Type":"ContainerStarted","Data":"c49b0ac4366f6c37623bd1d212460dd327e547328f43db84172cdd9f39fb2bd7"} Jul 10 18:18:39.146737 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:39.146317 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" event={"ID":"614d7858-1ef3-43d2-92ee-b0479f86d8fc","Type":"ContainerStarted","Data":"40d7ce321c45dbf7dbc309ffb2dd3e1a9e269536472c0d47d1e614684d8d175e"} Jul 10 18:18:39.147720 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:39.147693 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9rqql" event={"ID":"91c02392-14bc-403d-ab9f-131ed9213d28","Type":"ContainerStarted","Data":"bd194419fd6c828f1cc8eed86df0af926fff6dbd8bda945c6f3612ebfab4ebdb"} Jul 10 18:18:39.149112 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:39.149088 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d86479bb-kvth6" event={"ID":"0bb8f3dc-1669-4a8c-8b1f-9486cb198294","Type":"ContainerStarted","Data":"4eb05dd0a54310b4268a276007e918079ee6c62aa2682618e1b42a8307eba5cf"} Jul 10 18:18:39.150382 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:39.150360 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lftkj" event={"ID":"0a36554d-7c38-4893-9dca-ceac16436f2e","Type":"ContainerStarted","Data":"e9b0cb2ac82efdad18430b37e4ae49a55b77be632034e183b74814dcace129e5"} Jul 10 18:18:39.150486 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:39.150412 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:18:39.152000 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:39.151984 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xksd4" event={"ID":"2f17d7a9-1710-4246-9196-6c85ce329679","Type":"ContainerStarted","Data":"afdecaf6e3799a6c988b3d2d56da30a53135a97ad3bbc37a595bd78b54658527"} Jul 10 18:18:39.152081 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:39.152016 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xksd4" event={"ID":"2f17d7a9-1710-4246-9196-6c85ce329679","Type":"ContainerStarted","Data":"571adb827d264431b97f08d5e306993c36b65817ca79e0fe64f487b926a76f49"} Jul 10 18:18:39.152081 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:39.152051 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xksd4" Jul 10 18:18:39.173600 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:39.173540 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d4548c964-p8crn" podStartSLOduration=1.8613460160000002 podStartE2EDuration="10.173518769s" totalImagesPullingTime="8.312172753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:39.172197373 +0000 UTC m=+76.071466481" watchObservedRunningTime="2026-07-10 18:18:39.173518769 +0000 UTC m=+76.072787879" Jul 10 18:18:39.174042 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:39.174016 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ndsqx" podStartSLOduration=74.877937241 podStartE2EDuration="1m16.174006957s" totalImagesPullingTime="1.296069716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:32.135907297 +0000 UTC m=+69.035176406" watchObservedRunningTime="2026-07-10 18:18:39.174006957 +0000 UTC m=+76.073276105" Jul 10 18:18:39.197958 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:39.197904 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9rqql" podStartSLOduration=67.467657202 podStartE2EDuration="1m16.197887039s" totalImagesPullingTime="8.730229837s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:39.197629679 +0000 UTC m=+76.096898778" watchObservedRunningTime="2026-07-10 18:18:39.197887039 +0000 UTC m=+76.097156146" Jul 10 18:18:39.225050 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:39.224995 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lftkj" podStartSLOduration=67.787838544 podStartE2EDuration="1m16.224978209s" totalImagesPullingTime="8.437139665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:39.224583624 +0000 UTC m=+76.123852731" watchObservedRunningTime="2026-07-10 18:18:39.224978209 +0000 UTC m=+76.124247316" Jul 10 18:18:39.258133 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:39.258079 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xksd4" podStartSLOduration=34.340020678 podStartE2EDuration="41.258062984s" totalImagesPullingTime="6.918042306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:17:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:39.257070089 +0000 UTC m=+76.156339221" watchObservedRunningTime="2026-07-10 18:18:39.258062984 +0000 UTC m=+76.157332153" Jul 10 18:18:39.290780 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:39.290722 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57d86479bb-kvth6" podStartSLOduration=1.9724125209999999 podStartE2EDuration="10.290702379s" totalImagesPullingTime="8.318289858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:39.289208455 +0000 UTC m=+76.188477564" watchObservedRunningTime="2026-07-10 18:18:39.290702379 +0000 UTC m=+76.189971488" Jul 10 18:18:42.091498 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.091463 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2rz6f"] Jul 10 18:18:42.096694 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.096663 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.099698 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.099670 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 18:18:42.100012 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.099991 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 18:18:42.100012 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.100006 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 18:18:42.100098 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.100005 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 18:18:42.100918 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.100902 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 18:18:42.101035 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.101018 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 18:18:42.101078 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.101020 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zjc65\"" Jul 10 18:18:42.194909 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.194868 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b997f6d-4391-4ecb-bb62-125e3a906d63-metrics-client-ca\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.194909 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.194916 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b997f6d-4391-4ecb-bb62-125e3a906d63-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.195144 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.194993 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8b997f6d-4391-4ecb-bb62-125e3a906d63-node-exporter-wtmp\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.195144 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.195023 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbtt8\" (UniqueName: \"kubernetes.io/projected/8b997f6d-4391-4ecb-bb62-125e3a906d63-kube-api-access-vbtt8\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.195144 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.195057 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8b997f6d-4391-4ecb-bb62-125e3a906d63-node-exporter-accelerators-collector-config\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.195144 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.195086 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8b997f6d-4391-4ecb-bb62-125e3a906d63-root\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.195144 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.195127 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b997f6d-4391-4ecb-bb62-125e3a906d63-sys\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.195365 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.195229 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8b997f6d-4391-4ecb-bb62-125e3a906d63-node-exporter-tls\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.195365 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.195264 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8b997f6d-4391-4ecb-bb62-125e3a906d63-node-exporter-textfile\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.296123 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.296089 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8b997f6d-4391-4ecb-bb62-125e3a906d63-node-exporter-textfile\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.296123 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.296125 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b997f6d-4391-4ecb-bb62-125e3a906d63-metrics-client-ca\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.296368 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.296143 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b997f6d-4391-4ecb-bb62-125e3a906d63-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.296368 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.296169 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8b997f6d-4391-4ecb-bb62-125e3a906d63-node-exporter-wtmp\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.296368 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.296314 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8b997f6d-4391-4ecb-bb62-125e3a906d63-node-exporter-wtmp\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.296522 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.296371 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbtt8\" (UniqueName: \"kubernetes.io/projected/8b997f6d-4391-4ecb-bb62-125e3a906d63-kube-api-access-vbtt8\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.296522 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.296440 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8b997f6d-4391-4ecb-bb62-125e3a906d63-node-exporter-accelerators-collector-config\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.296522 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.296487 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8b997f6d-4391-4ecb-bb62-125e3a906d63-root\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.296522 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.296499 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8b997f6d-4391-4ecb-bb62-125e3a906d63-node-exporter-textfile\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.296522 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.296515 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b997f6d-4391-4ecb-bb62-125e3a906d63-sys\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.296791 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.296567 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8b997f6d-4391-4ecb-bb62-125e3a906d63-node-exporter-tls\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.296791 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.296671 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8b997f6d-4391-4ecb-bb62-125e3a906d63-root\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.296791 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.296718 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b997f6d-4391-4ecb-bb62-125e3a906d63-sys\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.296947 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.296828 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b997f6d-4391-4ecb-bb62-125e3a906d63-metrics-client-ca\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.297035 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.297015 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8b997f6d-4391-4ecb-bb62-125e3a906d63-node-exporter-accelerators-collector-config\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.298778 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.298751 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b997f6d-4391-4ecb-bb62-125e3a906d63-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.298891 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.298874 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8b997f6d-4391-4ecb-bb62-125e3a906d63-node-exporter-tls\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.306203 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.306181 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbtt8\" (UniqueName: \"kubernetes.io/projected/8b997f6d-4391-4ecb-bb62-125e3a906d63-kube-api-access-vbtt8\") pod \"node-exporter-2rz6f\" (UID: \"8b997f6d-4391-4ecb-bb62-125e3a906d63\") " pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:42.406240 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:42.406140 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2rz6f" Jul 10 18:18:43.169090 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:43.169044 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2rz6f" event={"ID":"8b997f6d-4391-4ecb-bb62-125e3a906d63","Type":"ContainerStarted","Data":"416fb4d592a48992da6fd4b9afa605e4a1ae582aa4a490a16476f2f5e6bd9a46"} Jul 10 18:18:44.173939 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:44.173904 2571 generic.go:356] "Generic (PLEG): container finished" podID="8b997f6d-4391-4ecb-bb62-125e3a906d63" containerID="c60aac5005fca7d54f2cc095a0038ede4b4ee816047a483dc800b91fe5a83eb0" exitCode=0 Jul 10 18:18:44.174435 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:44.173960 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2rz6f" event={"ID":"8b997f6d-4391-4ecb-bb62-125e3a906d63","Type":"ContainerDied","Data":"c60aac5005fca7d54f2cc095a0038ede4b4ee816047a483dc800b91fe5a83eb0"} Jul 10 18:18:45.179982 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:45.179869 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2rz6f" event={"ID":"8b997f6d-4391-4ecb-bb62-125e3a906d63","Type":"ContainerStarted","Data":"5af639d8b355ec73bedf8f3c5aabe3c8d9f465b4ea5eefaf816ab9a23e7fc2c4"} Jul 10 18:18:45.179982 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:45.179919 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2rz6f" event={"ID":"8b997f6d-4391-4ecb-bb62-125e3a906d63","Type":"ContainerStarted","Data":"025190afa5fd96b772e635b7086eefee19ad72a23be39b16616834d944206dd0"} Jul 10 18:18:45.203547 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:45.203490 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2rz6f" podStartSLOduration=2.219632331 podStartE2EDuration="3.203472063s" totalImagesPullingTime="919.268243ms" totalInitContainerRuntime="64.571489ms" isStatefulPod=true podCreationTimestamp="2026-07-10 18:18:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:18:45.202298419 +0000 UTC m=+82.101567518" watchObservedRunningTime="2026-07-10 18:18:45.203472063 +0000 UTC m=+82.102741168" Jul 10 18:18:49.158655 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:18:49.158599 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xksd4" Jul 10 18:19:10.158797 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:19:10.158766 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lftkj" Jul 10 18:22:23.489888 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:23.489853 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r9vz_0cafad48-3998-465d-b4c8-b5d94cac6b3c/ovn-acl-logging/0.log" Jul 10 18:22:23.490657 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:23.490642 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r9vz_0cafad48-3998-465d-b4c8-b5d94cac6b3c/ovn-acl-logging/0.log" Jul 10 18:22:23.493019 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:23.492999 2571 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 18:22:42.962185 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:42.962147 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9468p"] Jul 10 18:22:42.964192 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:42.964174 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9468p" Jul 10 18:22:42.969145 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:42.966641 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-zgdjj\"" Jul 10 18:22:42.969145 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:42.966967 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 10 18:22:42.969145 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:42.967226 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 10 18:22:42.979113 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:42.979051 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9468p"] Jul 10 18:22:43.058958 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:43.058922 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdgkd\" (UniqueName: \"kubernetes.io/projected/25b78742-8721-42e1-8578-e47b29769843-kube-api-access-qdgkd\") pod \"cert-manager-operator-controller-manager-55c444bf86-9468p\" (UID: \"25b78742-8721-42e1-8578-e47b29769843\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9468p" Jul 10 18:22:43.059148 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:43.058993 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25b78742-8721-42e1-8578-e47b29769843-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9468p\" (UID: \"25b78742-8721-42e1-8578-e47b29769843\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9468p" Jul 10 18:22:43.160127 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:43.160100 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdgkd\" (UniqueName: \"kubernetes.io/projected/25b78742-8721-42e1-8578-e47b29769843-kube-api-access-qdgkd\") pod \"cert-manager-operator-controller-manager-55c444bf86-9468p\" (UID: \"25b78742-8721-42e1-8578-e47b29769843\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9468p" Jul 10 18:22:43.160263 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:43.160159 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25b78742-8721-42e1-8578-e47b29769843-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9468p\" (UID: \"25b78742-8721-42e1-8578-e47b29769843\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9468p" Jul 10 18:22:43.160478 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:43.160464 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25b78742-8721-42e1-8578-e47b29769843-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9468p\" (UID: \"25b78742-8721-42e1-8578-e47b29769843\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9468p" Jul 10 18:22:43.168487 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:43.168458 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdgkd\" (UniqueName: \"kubernetes.io/projected/25b78742-8721-42e1-8578-e47b29769843-kube-api-access-qdgkd\") pod \"cert-manager-operator-controller-manager-55c444bf86-9468p\" (UID: \"25b78742-8721-42e1-8578-e47b29769843\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9468p" Jul 10 18:22:43.273963 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:43.273921 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9468p" Jul 10 18:22:43.409045 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:43.409004 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9468p"] Jul 10 18:22:43.413211 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:22:43.413185 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25b78742_8721_42e1_8578_e47b29769843.slice/crio-c775de027ae726b7458ac8157b19584da0e4196702499f4b737dbb9598599cab WatchSource:0}: Error finding container c775de027ae726b7458ac8157b19584da0e4196702499f4b737dbb9598599cab: Status 404 returned error can't find the container with id c775de027ae726b7458ac8157b19584da0e4196702499f4b737dbb9598599cab Jul 10 18:22:43.415901 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:43.415883 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:22:43.974021 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:43.973983 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9468p" event={"ID":"25b78742-8721-42e1-8578-e47b29769843","Type":"ContainerStarted","Data":"c775de027ae726b7458ac8157b19584da0e4196702499f4b737dbb9598599cab"} Jul 10 18:22:46.986817 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:46.986783 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9468p" event={"ID":"25b78742-8721-42e1-8578-e47b29769843","Type":"ContainerStarted","Data":"03e1be7c4a2ca9132d995de6b120c38cdedcd4f62dafbb97b1d0a3d8d9ed24c4"} Jul 10 18:22:47.041311 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:47.041260 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9468p" podStartSLOduration=2.510367768 podStartE2EDuration="5.041239247s" totalImagesPullingTime="2.530871479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:22:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:22:47.03886825 +0000 UTC m=+323.938137358" watchObservedRunningTime="2026-07-10 18:22:47.041239247 +0000 UTC m=+323.940508354" Jul 10 18:22:47.657259 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:47.657227 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw"] Jul 10 18:22:47.659216 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:47.659199 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw" Jul 10 18:22:47.661797 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:47.661777 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 18:22:47.661903 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:47.661825 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 18:22:47.663330 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:47.663316 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-px9fh\"" Jul 10 18:22:47.673178 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:47.673158 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw"] Jul 10 18:22:47.699432 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:47.699400 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/91a8129c-972a-4d7c-af31-3d978307c35b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw\" (UID: \"91a8129c-972a-4d7c-af31-3d978307c35b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw" Jul 10 18:22:47.699432 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:47.699438 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/91a8129c-972a-4d7c-af31-3d978307c35b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw\" (UID: \"91a8129c-972a-4d7c-af31-3d978307c35b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw" Jul 10 18:22:47.699654 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:47.699474 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kl6s\" (UniqueName: \"kubernetes.io/projected/91a8129c-972a-4d7c-af31-3d978307c35b-kube-api-access-2kl6s\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw\" (UID: \"91a8129c-972a-4d7c-af31-3d978307c35b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw" Jul 10 18:22:47.800672 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:47.800605 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/91a8129c-972a-4d7c-af31-3d978307c35b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw\" (UID: \"91a8129c-972a-4d7c-af31-3d978307c35b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw" Jul 10 18:22:47.800869 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:47.800686 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/91a8129c-972a-4d7c-af31-3d978307c35b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw\" (UID: \"91a8129c-972a-4d7c-af31-3d978307c35b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw" Jul 10 18:22:47.800869 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:47.800745 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kl6s\" (UniqueName: \"kubernetes.io/projected/91a8129c-972a-4d7c-af31-3d978307c35b-kube-api-access-2kl6s\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw\" (UID: \"91a8129c-972a-4d7c-af31-3d978307c35b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw" Jul 10 18:22:47.801094 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:47.801070 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/91a8129c-972a-4d7c-af31-3d978307c35b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw\" (UID: \"91a8129c-972a-4d7c-af31-3d978307c35b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw" Jul 10 18:22:47.801132 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:47.801082 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/91a8129c-972a-4d7c-af31-3d978307c35b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw\" (UID: \"91a8129c-972a-4d7c-af31-3d978307c35b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw" Jul 10 18:22:47.818746 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:47.818712 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kl6s\" (UniqueName: \"kubernetes.io/projected/91a8129c-972a-4d7c-af31-3d978307c35b-kube-api-access-2kl6s\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw\" (UID: \"91a8129c-972a-4d7c-af31-3d978307c35b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw" Jul 10 18:22:47.967866 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:47.967767 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw" Jul 10 18:22:48.097527 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:48.097500 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw"] Jul 10 18:22:48.099746 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:22:48.099718 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91a8129c_972a_4d7c_af31_3d978307c35b.slice/crio-d966daab25b6a5f6eeb4c081b3c89998cbb68277c79d98f8653c84113706f2e7 WatchSource:0}: Error finding container d966daab25b6a5f6eeb4c081b3c89998cbb68277c79d98f8653c84113706f2e7: Status 404 returned error can't find the container with id d966daab25b6a5f6eeb4c081b3c89998cbb68277c79d98f8653c84113706f2e7 Jul 10 18:22:48.995870 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:48.995831 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw" event={"ID":"91a8129c-972a-4d7c-af31-3d978307c35b","Type":"ContainerStarted","Data":"d966daab25b6a5f6eeb4c081b3c89998cbb68277c79d98f8653c84113706f2e7"} Jul 10 18:22:54.332115 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:54.332084 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mxsxp"] Jul 10 18:22:54.335090 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:54.335068 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mxsxp" Jul 10 18:22:54.337844 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:54.337824 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 10 18:22:54.337954 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:54.337918 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 10 18:22:54.339182 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:54.339002 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-28tql\"" Jul 10 18:22:54.346776 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:54.346753 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mxsxp"] Jul 10 18:22:54.457661 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:54.457599 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrvqs\" (UniqueName: \"kubernetes.io/projected/ce3435de-f944-4fef-85ca-fc8f9e4ba28a-kube-api-access-mrvqs\") pod \"cert-manager-webhook-6486ffd997-mxsxp\" (UID: \"ce3435de-f944-4fef-85ca-fc8f9e4ba28a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mxsxp" Jul 10 18:22:54.457840 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:54.457694 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ce3435de-f944-4fef-85ca-fc8f9e4ba28a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mxsxp\" (UID: \"ce3435de-f944-4fef-85ca-fc8f9e4ba28a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mxsxp" Jul 10 18:22:54.558475 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:54.558435 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrvqs\" (UniqueName: \"kubernetes.io/projected/ce3435de-f944-4fef-85ca-fc8f9e4ba28a-kube-api-access-mrvqs\") pod \"cert-manager-webhook-6486ffd997-mxsxp\" (UID: \"ce3435de-f944-4fef-85ca-fc8f9e4ba28a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mxsxp" Jul 10 18:22:54.558664 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:54.558483 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ce3435de-f944-4fef-85ca-fc8f9e4ba28a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mxsxp\" (UID: \"ce3435de-f944-4fef-85ca-fc8f9e4ba28a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mxsxp" Jul 10 18:22:54.567572 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:54.567530 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ce3435de-f944-4fef-85ca-fc8f9e4ba28a-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mxsxp\" (UID: \"ce3435de-f944-4fef-85ca-fc8f9e4ba28a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mxsxp" Jul 10 18:22:54.567727 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:54.567702 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrvqs\" (UniqueName: \"kubernetes.io/projected/ce3435de-f944-4fef-85ca-fc8f9e4ba28a-kube-api-access-mrvqs\") pod \"cert-manager-webhook-6486ffd997-mxsxp\" (UID: \"ce3435de-f944-4fef-85ca-fc8f9e4ba28a\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mxsxp" Jul 10 18:22:54.645598 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:54.645513 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mxsxp" Jul 10 18:22:55.239058 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:55.239032 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mxsxp"] Jul 10 18:22:55.242196 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:22:55.242171 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce3435de_f944_4fef_85ca_fc8f9e4ba28a.slice/crio-6216c060c7e0b788861af8d0eb4461b62acb23b34231ef89cf634ca45a7c09b9 WatchSource:0}: Error finding container 6216c060c7e0b788861af8d0eb4461b62acb23b34231ef89cf634ca45a7c09b9: Status 404 returned error can't find the container with id 6216c060c7e0b788861af8d0eb4461b62acb23b34231ef89cf634ca45a7c09b9 Jul 10 18:22:56.025876 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:56.025836 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mxsxp" event={"ID":"ce3435de-f944-4fef-85ca-fc8f9e4ba28a","Type":"ContainerStarted","Data":"6216c060c7e0b788861af8d0eb4461b62acb23b34231ef89cf634ca45a7c09b9"} Jul 10 18:22:56.027760 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:56.027712 2571 generic.go:356] "Generic (PLEG): container finished" podID="91a8129c-972a-4d7c-af31-3d978307c35b" containerID="2f3cb675ef0fce6a95e73570abd5ce9c2ddfe3f468a233963a1b2311faa2309b" exitCode=0 Jul 10 18:22:56.027903 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:56.027806 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw" event={"ID":"91a8129c-972a-4d7c-af31-3d978307c35b","Type":"ContainerDied","Data":"2f3cb675ef0fce6a95e73570abd5ce9c2ddfe3f468a233963a1b2311faa2309b"} Jul 10 18:22:59.040985 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:59.040940 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mxsxp" event={"ID":"ce3435de-f944-4fef-85ca-fc8f9e4ba28a","Type":"ContainerStarted","Data":"66ae468c8ebc2cc439d0dccc591f50aded1f164c0ec4f4f76fe48563eada8574"} Jul 10 18:22:59.041442 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:59.040997 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mxsxp" Jul 10 18:22:59.042559 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:59.042540 2571 generic.go:356] "Generic (PLEG): container finished" podID="91a8129c-972a-4d7c-af31-3d978307c35b" containerID="a58f48c41896ceca9f8de2c927e46d7454b0f4a1275d3235ea1e13b084aad646" exitCode=0 Jul 10 18:22:59.042632 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:59.042593 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw" event={"ID":"91a8129c-972a-4d7c-af31-3d978307c35b","Type":"ContainerDied","Data":"a58f48c41896ceca9f8de2c927e46d7454b0f4a1275d3235ea1e13b084aad646"} Jul 10 18:22:59.064491 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:22:59.064424 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-mxsxp" podStartSLOduration=1.835489707 podStartE2EDuration="5.064403178s" totalImagesPullingTime="3.228913471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:22:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:22:59.064219553 +0000 UTC m=+335.963488669" watchObservedRunningTime="2026-07-10 18:22:59.064403178 +0000 UTC m=+335.963672274" Jul 10 18:23:05.049232 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:05.049191 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mxsxp" Jul 10 18:23:07.075442 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:07.075407 2571 generic.go:356] "Generic (PLEG): container finished" podID="91a8129c-972a-4d7c-af31-3d978307c35b" containerID="fe075e6029ef7bc94919cabf3e30014ea77b7c2a2d18db6745a353d00a02ee48" exitCode=0 Jul 10 18:23:07.075849 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:07.075496 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw" event={"ID":"91a8129c-972a-4d7c-af31-3d978307c35b","Type":"ContainerDied","Data":"fe075e6029ef7bc94919cabf3e30014ea77b7c2a2d18db6745a353d00a02ee48"} Jul 10 18:23:08.207054 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:08.207028 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw" Jul 10 18:23:08.373224 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:08.373126 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/91a8129c-972a-4d7c-af31-3d978307c35b-util\") pod \"91a8129c-972a-4d7c-af31-3d978307c35b\" (UID: \"91a8129c-972a-4d7c-af31-3d978307c35b\") " Jul 10 18:23:08.373224 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:08.373174 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/91a8129c-972a-4d7c-af31-3d978307c35b-bundle\") pod \"91a8129c-972a-4d7c-af31-3d978307c35b\" (UID: \"91a8129c-972a-4d7c-af31-3d978307c35b\") " Jul 10 18:23:08.373224 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:08.373227 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kl6s\" (UniqueName: \"kubernetes.io/projected/91a8129c-972a-4d7c-af31-3d978307c35b-kube-api-access-2kl6s\") pod \"91a8129c-972a-4d7c-af31-3d978307c35b\" (UID: \"91a8129c-972a-4d7c-af31-3d978307c35b\") " Jul 10 18:23:08.373604 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:08.373576 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91a8129c-972a-4d7c-af31-3d978307c35b-bundle" (OuterVolumeSpecName: "bundle") pod "91a8129c-972a-4d7c-af31-3d978307c35b" (UID: "91a8129c-972a-4d7c-af31-3d978307c35b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:23:08.375421 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:08.375390 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91a8129c-972a-4d7c-af31-3d978307c35b-kube-api-access-2kl6s" (OuterVolumeSpecName: "kube-api-access-2kl6s") pod "91a8129c-972a-4d7c-af31-3d978307c35b" (UID: "91a8129c-972a-4d7c-af31-3d978307c35b"). InnerVolumeSpecName "kube-api-access-2kl6s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:23:08.377882 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:08.377854 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91a8129c-972a-4d7c-af31-3d978307c35b-util" (OuterVolumeSpecName: "util") pod "91a8129c-972a-4d7c-af31-3d978307c35b" (UID: "91a8129c-972a-4d7c-af31-3d978307c35b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:23:08.474400 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:08.474367 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/91a8129c-972a-4d7c-af31-3d978307c35b-bundle\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:23:08.474400 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:08.474394 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kl6s\" (UniqueName: \"kubernetes.io/projected/91a8129c-972a-4d7c-af31-3d978307c35b-kube-api-access-2kl6s\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:23:08.474400 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:08.474404 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/91a8129c-972a-4d7c-af31-3d978307c35b-util\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:23:09.084551 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:09.084524 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw" Jul 10 18:23:09.084724 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:09.084521 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4r7lw" event={"ID":"91a8129c-972a-4d7c-af31-3d978307c35b","Type":"ContainerDied","Data":"d966daab25b6a5f6eeb4c081b3c89998cbb68277c79d98f8653c84113706f2e7"} Jul 10 18:23:09.084724 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:09.084672 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d966daab25b6a5f6eeb4c081b3c89998cbb68277c79d98f8653c84113706f2e7" Jul 10 18:23:34.624804 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.624760 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2"] Jul 10 18:23:34.625416 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.625110 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91a8129c-972a-4d7c-af31-3d978307c35b" containerName="extract" Jul 10 18:23:34.625416 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.625127 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="91a8129c-972a-4d7c-af31-3d978307c35b" containerName="extract" Jul 10 18:23:34.625416 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.625148 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91a8129c-972a-4d7c-af31-3d978307c35b" containerName="util" Jul 10 18:23:34.625416 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.625158 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="91a8129c-972a-4d7c-af31-3d978307c35b" containerName="util" Jul 10 18:23:34.625416 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.625166 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91a8129c-972a-4d7c-af31-3d978307c35b" containerName="pull" Jul 10 18:23:34.625416 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.625174 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="91a8129c-972a-4d7c-af31-3d978307c35b" containerName="pull" Jul 10 18:23:34.625416 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.625233 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="91a8129c-972a-4d7c-af31-3d978307c35b" containerName="extract" Jul 10 18:23:34.627290 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.627267 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" Jul 10 18:23:34.631035 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.631011 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 18:23:34.631035 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.631031 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5dt62\"" Jul 10 18:23:34.631207 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.631013 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 18:23:34.631405 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.631390 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 18:23:34.631453 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.631391 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 18:23:34.644318 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.644290 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2"] Jul 10 18:23:34.661624 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.661577 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/254586fb-ea8a-4085-97da-1a3c5c95454d-tmp\") pod \"opendatahub-operator-controller-manager-6d75465dc8-27qb2\" (UID: \"254586fb-ea8a-4085-97da-1a3c5c95454d\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" Jul 10 18:23:34.661806 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.661645 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/254586fb-ea8a-4085-97da-1a3c5c95454d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d75465dc8-27qb2\" (UID: \"254586fb-ea8a-4085-97da-1a3c5c95454d\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" Jul 10 18:23:34.661806 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.661668 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/254586fb-ea8a-4085-97da-1a3c5c95454d-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d75465dc8-27qb2\" (UID: \"254586fb-ea8a-4085-97da-1a3c5c95454d\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" Jul 10 18:23:34.661806 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.661687 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xm4r\" (UniqueName: \"kubernetes.io/projected/254586fb-ea8a-4085-97da-1a3c5c95454d-kube-api-access-6xm4r\") pod \"opendatahub-operator-controller-manager-6d75465dc8-27qb2\" (UID: \"254586fb-ea8a-4085-97da-1a3c5c95454d\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" Jul 10 18:23:34.661806 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.661724 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/254586fb-ea8a-4085-97da-1a3c5c95454d-manifests\") pod \"opendatahub-operator-controller-manager-6d75465dc8-27qb2\" (UID: \"254586fb-ea8a-4085-97da-1a3c5c95454d\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" Jul 10 18:23:34.762677 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.762637 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/254586fb-ea8a-4085-97da-1a3c5c95454d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d75465dc8-27qb2\" (UID: \"254586fb-ea8a-4085-97da-1a3c5c95454d\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" Jul 10 18:23:34.762848 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.762696 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/254586fb-ea8a-4085-97da-1a3c5c95454d-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d75465dc8-27qb2\" (UID: \"254586fb-ea8a-4085-97da-1a3c5c95454d\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" Jul 10 18:23:34.762848 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.762718 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xm4r\" (UniqueName: \"kubernetes.io/projected/254586fb-ea8a-4085-97da-1a3c5c95454d-kube-api-access-6xm4r\") pod \"opendatahub-operator-controller-manager-6d75465dc8-27qb2\" (UID: \"254586fb-ea8a-4085-97da-1a3c5c95454d\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" Jul 10 18:23:34.762848 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.762747 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/254586fb-ea8a-4085-97da-1a3c5c95454d-manifests\") pod \"opendatahub-operator-controller-manager-6d75465dc8-27qb2\" (UID: \"254586fb-ea8a-4085-97da-1a3c5c95454d\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" Jul 10 18:23:34.762949 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.762898 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/254586fb-ea8a-4085-97da-1a3c5c95454d-tmp\") pod \"opendatahub-operator-controller-manager-6d75465dc8-27qb2\" (UID: \"254586fb-ea8a-4085-97da-1a3c5c95454d\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" Jul 10 18:23:34.763139 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.763116 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/254586fb-ea8a-4085-97da-1a3c5c95454d-manifests\") pod \"opendatahub-operator-controller-manager-6d75465dc8-27qb2\" (UID: \"254586fb-ea8a-4085-97da-1a3c5c95454d\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" Jul 10 18:23:34.764970 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.764946 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/254586fb-ea8a-4085-97da-1a3c5c95454d-tmp\") pod \"opendatahub-operator-controller-manager-6d75465dc8-27qb2\" (UID: \"254586fb-ea8a-4085-97da-1a3c5c95454d\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" Jul 10 18:23:34.765073 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.764993 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/254586fb-ea8a-4085-97da-1a3c5c95454d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d75465dc8-27qb2\" (UID: \"254586fb-ea8a-4085-97da-1a3c5c95454d\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" Jul 10 18:23:34.765112 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.765066 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/254586fb-ea8a-4085-97da-1a3c5c95454d-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d75465dc8-27qb2\" (UID: \"254586fb-ea8a-4085-97da-1a3c5c95454d\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" Jul 10 18:23:34.773593 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.773573 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xm4r\" (UniqueName: \"kubernetes.io/projected/254586fb-ea8a-4085-97da-1a3c5c95454d-kube-api-access-6xm4r\") pod \"opendatahub-operator-controller-manager-6d75465dc8-27qb2\" (UID: \"254586fb-ea8a-4085-97da-1a3c5c95454d\") " pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" Jul 10 18:23:34.936909 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:34.936810 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" Jul 10 18:23:35.067008 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:35.066976 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2"] Jul 10 18:23:35.070626 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:23:35.070582 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod254586fb_ea8a_4085_97da_1a3c5c95454d.slice/crio-b65c466cd37b51a3b97d093199003efdf51c99985653a5dbfbdbde62a93e8342 WatchSource:0}: Error finding container b65c466cd37b51a3b97d093199003efdf51c99985653a5dbfbdbde62a93e8342: Status 404 returned error can't find the container with id b65c466cd37b51a3b97d093199003efdf51c99985653a5dbfbdbde62a93e8342 Jul 10 18:23:35.180042 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:35.180007 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" event={"ID":"254586fb-ea8a-4085-97da-1a3c5c95454d","Type":"ContainerStarted","Data":"b65c466cd37b51a3b97d093199003efdf51c99985653a5dbfbdbde62a93e8342"} Jul 10 18:23:39.203432 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:39.203392 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" event={"ID":"254586fb-ea8a-4085-97da-1a3c5c95454d","Type":"ContainerStarted","Data":"b9cb1b219f6fee9778a1e4d0e569b83688a398d41cde5b73534f7f65822e4328"} Jul 10 18:23:40.209008 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:40.208913 2571 generic.go:356] "Generic (PLEG): container finished" podID="254586fb-ea8a-4085-97da-1a3c5c95454d" containerID="b9cb1b219f6fee9778a1e4d0e569b83688a398d41cde5b73534f7f65822e4328" exitCode=0 Jul 10 18:23:40.209433 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:40.209000 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" event={"ID":"254586fb-ea8a-4085-97da-1a3c5c95454d","Type":"ContainerDied","Data":"b9cb1b219f6fee9778a1e4d0e569b83688a398d41cde5b73534f7f65822e4328"} Jul 10 18:23:41.215856 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:41.215818 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" event={"ID":"254586fb-ea8a-4085-97da-1a3c5c95454d","Type":"ContainerStarted","Data":"2db3aabae8b52a312080c1fde40fd9fbeb88beca81ce05a519acb0ad4b3ea59f"} Jul 10 18:23:41.216260 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:41.215900 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" Jul 10 18:23:41.239115 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:41.239054 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" podStartSLOduration=2.211685623 podStartE2EDuration="7.239035315s" totalImagesPullingTime="4.022153843s" totalInitContainerRuntime="1.005195849s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:23:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:23:41.237726981 +0000 UTC m=+378.136996109" watchObservedRunningTime="2026-07-10 18:23:41.239035315 +0000 UTC m=+378.138304423" Jul 10 18:23:52.221895 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:52.221866 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6d75465dc8-27qb2" Jul 10 18:23:54.942359 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:54.942320 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-79bb7c45b8-ss6xx"] Jul 10 18:23:54.945463 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:54.945441 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-ss6xx" Jul 10 18:23:54.949427 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:54.949403 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 10 18:23:54.949763 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:54.949745 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-mgsjt\"" Jul 10 18:23:54.950234 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:54.950214 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 10 18:23:54.953596 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:54.953576 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 18:23:54.953723 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:54.953689 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 18:23:54.969916 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:54.969894 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-79bb7c45b8-ss6xx"] Jul 10 18:23:54.996909 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:54.996879 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-j46xf"] Jul 10 18:23:54.999919 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:54.999902 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-j46xf" Jul 10 18:23:55.009008 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.008985 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 10 18:23:55.009148 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.009072 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-2c29n\"" Jul 10 18:23:55.011713 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.011695 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9764f143-44c0-4790-8b60-530bd022797f-tmp\") pod \"kube-auth-proxy-79bb7c45b8-ss6xx\" (UID: \"9764f143-44c0-4790-8b60-530bd022797f\") " pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-ss6xx" Jul 10 18:23:55.011784 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.011756 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27jgt\" (UniqueName: \"kubernetes.io/projected/9764f143-44c0-4790-8b60-530bd022797f-kube-api-access-27jgt\") pod \"kube-auth-proxy-79bb7c45b8-ss6xx\" (UID: \"9764f143-44c0-4790-8b60-530bd022797f\") " pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-ss6xx" Jul 10 18:23:55.011824 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.011795 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9764f143-44c0-4790-8b60-530bd022797f-tls-certs\") pod \"kube-auth-proxy-79bb7c45b8-ss6xx\" (UID: \"9764f143-44c0-4790-8b60-530bd022797f\") " pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-ss6xx" Jul 10 18:23:55.019512 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.019487 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-j46xf"] Jul 10 18:23:55.112158 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.112125 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27jgt\" (UniqueName: \"kubernetes.io/projected/9764f143-44c0-4790-8b60-530bd022797f-kube-api-access-27jgt\") pod \"kube-auth-proxy-79bb7c45b8-ss6xx\" (UID: \"9764f143-44c0-4790-8b60-530bd022797f\") " pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-ss6xx" Jul 10 18:23:55.112337 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.112176 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9764f143-44c0-4790-8b60-530bd022797f-tls-certs\") pod \"kube-auth-proxy-79bb7c45b8-ss6xx\" (UID: \"9764f143-44c0-4790-8b60-530bd022797f\") " pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-ss6xx" Jul 10 18:23:55.112337 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.112227 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9764f143-44c0-4790-8b60-530bd022797f-tmp\") pod \"kube-auth-proxy-79bb7c45b8-ss6xx\" (UID: \"9764f143-44c0-4790-8b60-530bd022797f\") " pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-ss6xx" Jul 10 18:23:55.112337 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.112259 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfxbs\" (UniqueName: \"kubernetes.io/projected/da45c16f-fdef-4fd6-b301-bf1a78e557d7-kube-api-access-rfxbs\") pod \"odh-model-controller-7998c67dc4-j46xf\" (UID: \"da45c16f-fdef-4fd6-b301-bf1a78e557d7\") " pod="opendatahub/odh-model-controller-7998c67dc4-j46xf" Jul 10 18:23:55.112337 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.112299 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/da45c16f-fdef-4fd6-b301-bf1a78e557d7-cert\") pod \"odh-model-controller-7998c67dc4-j46xf\" (UID: \"da45c16f-fdef-4fd6-b301-bf1a78e557d7\") " pod="opendatahub/odh-model-controller-7998c67dc4-j46xf" Jul 10 18:23:55.112535 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:23:55.112352 2571 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 10 18:23:55.112535 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:23:55.112467 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9764f143-44c0-4790-8b60-530bd022797f-tls-certs podName:9764f143-44c0-4790-8b60-530bd022797f nodeName:}" failed. No retries permitted until 2026-07-10 18:23:55.612437707 +0000 UTC m=+392.511706798 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9764f143-44c0-4790-8b60-530bd022797f-tls-certs") pod "kube-auth-proxy-79bb7c45b8-ss6xx" (UID: "9764f143-44c0-4790-8b60-530bd022797f") : secret "kube-auth-proxy-tls" not found Jul 10 18:23:55.114413 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.114396 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9764f143-44c0-4790-8b60-530bd022797f-tmp\") pod \"kube-auth-proxy-79bb7c45b8-ss6xx\" (UID: \"9764f143-44c0-4790-8b60-530bd022797f\") " pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-ss6xx" Jul 10 18:23:55.129190 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.129163 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27jgt\" (UniqueName: \"kubernetes.io/projected/9764f143-44c0-4790-8b60-530bd022797f-kube-api-access-27jgt\") pod \"kube-auth-proxy-79bb7c45b8-ss6xx\" (UID: \"9764f143-44c0-4790-8b60-530bd022797f\") " pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-ss6xx" Jul 10 18:23:55.212908 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.212811 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfxbs\" (UniqueName: \"kubernetes.io/projected/da45c16f-fdef-4fd6-b301-bf1a78e557d7-kube-api-access-rfxbs\") pod \"odh-model-controller-7998c67dc4-j46xf\" (UID: \"da45c16f-fdef-4fd6-b301-bf1a78e557d7\") " pod="opendatahub/odh-model-controller-7998c67dc4-j46xf" Jul 10 18:23:55.212908 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.212865 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/da45c16f-fdef-4fd6-b301-bf1a78e557d7-cert\") pod \"odh-model-controller-7998c67dc4-j46xf\" (UID: \"da45c16f-fdef-4fd6-b301-bf1a78e557d7\") " pod="opendatahub/odh-model-controller-7998c67dc4-j46xf" Jul 10 18:23:55.213107 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:23:55.213043 2571 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 10 18:23:55.213150 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:23:55.213120 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da45c16f-fdef-4fd6-b301-bf1a78e557d7-cert podName:da45c16f-fdef-4fd6-b301-bf1a78e557d7 nodeName:}" failed. No retries permitted until 2026-07-10 18:23:55.713105218 +0000 UTC m=+392.612374304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/da45c16f-fdef-4fd6-b301-bf1a78e557d7-cert") pod "odh-model-controller-7998c67dc4-j46xf" (UID: "da45c16f-fdef-4fd6-b301-bf1a78e557d7") : secret "odh-model-controller-webhook-cert" not found Jul 10 18:23:55.222899 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.222872 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfxbs\" (UniqueName: \"kubernetes.io/projected/da45c16f-fdef-4fd6-b301-bf1a78e557d7-kube-api-access-rfxbs\") pod \"odh-model-controller-7998c67dc4-j46xf\" (UID: \"da45c16f-fdef-4fd6-b301-bf1a78e557d7\") " pod="opendatahub/odh-model-controller-7998c67dc4-j46xf" Jul 10 18:23:55.617094 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.617056 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9764f143-44c0-4790-8b60-530bd022797f-tls-certs\") pod \"kube-auth-proxy-79bb7c45b8-ss6xx\" (UID: \"9764f143-44c0-4790-8b60-530bd022797f\") " pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-ss6xx" Jul 10 18:23:55.619468 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.619440 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9764f143-44c0-4790-8b60-530bd022797f-tls-certs\") pod \"kube-auth-proxy-79bb7c45b8-ss6xx\" (UID: \"9764f143-44c0-4790-8b60-530bd022797f\") " pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-ss6xx" Jul 10 18:23:55.718103 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.718063 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/da45c16f-fdef-4fd6-b301-bf1a78e557d7-cert\") pod \"odh-model-controller-7998c67dc4-j46xf\" (UID: \"da45c16f-fdef-4fd6-b301-bf1a78e557d7\") " pod="opendatahub/odh-model-controller-7998c67dc4-j46xf" Jul 10 18:23:55.720466 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.720436 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/da45c16f-fdef-4fd6-b301-bf1a78e557d7-cert\") pod \"odh-model-controller-7998c67dc4-j46xf\" (UID: \"da45c16f-fdef-4fd6-b301-bf1a78e557d7\") " pod="opendatahub/odh-model-controller-7998c67dc4-j46xf" Jul 10 18:23:55.807142 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.807105 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mc5v5"] Jul 10 18:23:55.810462 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.810438 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-mc5v5" Jul 10 18:23:55.813680 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.813649 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 10 18:23:55.825111 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.825082 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mc5v5"] Jul 10 18:23:55.855208 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.855172 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-ss6xx" Jul 10 18:23:55.910146 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.910115 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-j46xf" Jul 10 18:23:55.919643 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.919308 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkn98\" (UniqueName: \"kubernetes.io/projected/7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d-kube-api-access-vkn98\") pod \"model-serving-api-54d89bd5fc-mc5v5\" (UID: \"7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d\") " pod="opendatahub/model-serving-api-54d89bd5fc-mc5v5" Jul 10 18:23:55.919643 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:55.919363 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d-tls-certs\") pod \"model-serving-api-54d89bd5fc-mc5v5\" (UID: \"7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d\") " pod="opendatahub/model-serving-api-54d89bd5fc-mc5v5" Jul 10 18:23:56.011763 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.011726 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-79bb7c45b8-ss6xx"] Jul 10 18:23:56.014295 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:23:56.014257 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9764f143_44c0_4790_8b60_530bd022797f.slice/crio-a3c59dd63609087995880d36407fab7565bafdf08cb2602935a37ff861c630a7 WatchSource:0}: Error finding container a3c59dd63609087995880d36407fab7565bafdf08cb2602935a37ff861c630a7: Status 404 returned error can't find the container with id a3c59dd63609087995880d36407fab7565bafdf08cb2602935a37ff861c630a7 Jul 10 18:23:56.019857 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.019832 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d-tls-certs\") pod \"model-serving-api-54d89bd5fc-mc5v5\" (UID: \"7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d\") " pod="opendatahub/model-serving-api-54d89bd5fc-mc5v5" Jul 10 18:23:56.019981 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.019885 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkn98\" (UniqueName: \"kubernetes.io/projected/7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d-kube-api-access-vkn98\") pod \"model-serving-api-54d89bd5fc-mc5v5\" (UID: \"7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d\") " pod="opendatahub/model-serving-api-54d89bd5fc-mc5v5" Jul 10 18:23:56.020068 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:23:56.020048 2571 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 10 18:23:56.020145 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:23:56.020131 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d-tls-certs podName:7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d nodeName:}" failed. No retries permitted until 2026-07-10 18:23:56.520105992 +0000 UTC m=+393.419375093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d-tls-certs") pod "model-serving-api-54d89bd5fc-mc5v5" (UID: "7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d") : secret "model-serving-api-tls" not found Jul 10 18:23:56.047516 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.047483 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkn98\" (UniqueName: \"kubernetes.io/projected/7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d-kube-api-access-vkn98\") pod \"model-serving-api-54d89bd5fc-mc5v5\" (UID: \"7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d\") " pod="opendatahub/model-serving-api-54d89bd5fc-mc5v5" Jul 10 18:23:56.080842 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.080812 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-j46xf"] Jul 10 18:23:56.083882 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:23:56.083853 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda45c16f_fdef_4fd6_b301_bf1a78e557d7.slice/crio-24ffb1c5b78881042dd6549332b285525946cf26527700e357775513965898d5 WatchSource:0}: Error finding container 24ffb1c5b78881042dd6549332b285525946cf26527700e357775513965898d5: Status 404 returned error can't find the container with id 24ffb1c5b78881042dd6549332b285525946cf26527700e357775513965898d5 Jul 10 18:23:56.281444 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.281401 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-j46xf" event={"ID":"da45c16f-fdef-4fd6-b301-bf1a78e557d7","Type":"ContainerStarted","Data":"24ffb1c5b78881042dd6549332b285525946cf26527700e357775513965898d5"} Jul 10 18:23:56.282743 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.282712 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-ss6xx" event={"ID":"9764f143-44c0-4790-8b60-530bd022797f","Type":"ContainerStarted","Data":"a3c59dd63609087995880d36407fab7565bafdf08cb2602935a37ff861c630a7"} Jul 10 18:23:56.333839 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.333803 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-545gn"] Jul 10 18:23:56.338090 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.338065 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.340828 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.340807 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 10 18:23:56.340965 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.340845 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-mfql5\"" Jul 10 18:23:56.341100 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.341083 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 18:23:56.341195 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.341084 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 10 18:23:56.341319 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.341307 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 10 18:23:56.348023 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.348002 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-545gn"] Jul 10 18:23:56.422874 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.422841 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-545gn\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.422874 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.422880 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-545gn\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.423113 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.422898 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9bmd\" (UniqueName: \"kubernetes.io/projected/08d3e61e-93fc-41cf-93f9-35655cfcbbac-kube-api-access-x9bmd\") pod \"istiod-openshift-gateway-7574459656-545gn\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.423113 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.422928 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-token\") pod \"istiod-openshift-gateway-7574459656-545gn\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.423113 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.422962 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/08d3e61e-93fc-41cf-93f9-35655cfcbbac-local-certs\") pod \"istiod-openshift-gateway-7574459656-545gn\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.423113 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.422980 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/08d3e61e-93fc-41cf-93f9-35655cfcbbac-cacerts\") pod \"istiod-openshift-gateway-7574459656-545gn\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.423113 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.423006 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-545gn\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.523744 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.523703 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-545gn\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.523744 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.523748 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-545gn\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.523984 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.523772 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9bmd\" (UniqueName: \"kubernetes.io/projected/08d3e61e-93fc-41cf-93f9-35655cfcbbac-kube-api-access-x9bmd\") pod \"istiod-openshift-gateway-7574459656-545gn\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.523984 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.523806 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d-tls-certs\") pod \"model-serving-api-54d89bd5fc-mc5v5\" (UID: \"7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d\") " pod="opendatahub/model-serving-api-54d89bd5fc-mc5v5" Jul 10 18:23:56.523984 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.523833 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-token\") pod \"istiod-openshift-gateway-7574459656-545gn\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.523984 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.523860 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/08d3e61e-93fc-41cf-93f9-35655cfcbbac-local-certs\") pod \"istiod-openshift-gateway-7574459656-545gn\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.524182 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.523994 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/08d3e61e-93fc-41cf-93f9-35655cfcbbac-cacerts\") pod \"istiod-openshift-gateway-7574459656-545gn\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.524182 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.524049 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-545gn\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.524963 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.524935 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-545gn\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.526464 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.526431 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/08d3e61e-93fc-41cf-93f9-35655cfcbbac-local-certs\") pod \"istiod-openshift-gateway-7574459656-545gn\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.526585 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.526514 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/08d3e61e-93fc-41cf-93f9-35655cfcbbac-cacerts\") pod \"istiod-openshift-gateway-7574459656-545gn\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.526585 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.526521 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d-tls-certs\") pod \"model-serving-api-54d89bd5fc-mc5v5\" (UID: \"7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d\") " pod="opendatahub/model-serving-api-54d89bd5fc-mc5v5" Jul 10 18:23:56.526751 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.526683 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-545gn\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.526811 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.526769 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-545gn\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.532814 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.532757 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-token\") pod \"istiod-openshift-gateway-7574459656-545gn\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.532896 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.532841 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9bmd\" (UniqueName: \"kubernetes.io/projected/08d3e61e-93fc-41cf-93f9-35655cfcbbac-kube-api-access-x9bmd\") pod \"istiod-openshift-gateway-7574459656-545gn\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.647626 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.647572 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:23:56.728005 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.727977 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-mc5v5" Jul 10 18:23:56.795029 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.794941 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-545gn"] Jul 10 18:23:56.798135 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:23:56.798091 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08d3e61e_93fc_41cf_93f9_35655cfcbbac.slice/crio-e6028ec81db536e91cd02e79ecd8842314fa6a402884d5bf409ca08ed61c81c9 WatchSource:0}: Error finding container e6028ec81db536e91cd02e79ecd8842314fa6a402884d5bf409ca08ed61c81c9: Status 404 returned error can't find the container with id e6028ec81db536e91cd02e79ecd8842314fa6a402884d5bf409ca08ed61c81c9 Jul 10 18:23:56.912262 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:56.912225 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mc5v5"] Jul 10 18:23:56.918817 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:23:56.918777 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7791dd5d_23fb_4d0f_98ae_e30e0dfcb94d.slice/crio-2b1c37a693d993c08d8d5b8c3a03670b39e54b83635771c5014e28839361ab8c WatchSource:0}: Error finding container 2b1c37a693d993c08d8d5b8c3a03670b39e54b83635771c5014e28839361ab8c: Status 404 returned error can't find the container with id 2b1c37a693d993c08d8d5b8c3a03670b39e54b83635771c5014e28839361ab8c Jul 10 18:23:57.291541 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:57.291497 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" event={"ID":"08d3e61e-93fc-41cf-93f9-35655cfcbbac","Type":"ContainerStarted","Data":"e6028ec81db536e91cd02e79ecd8842314fa6a402884d5bf409ca08ed61c81c9"} Jul 10 18:23:57.292962 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:23:57.292894 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-mc5v5" event={"ID":"7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d","Type":"ContainerStarted","Data":"2b1c37a693d993c08d8d5b8c3a03670b39e54b83635771c5014e28839361ab8c"} Jul 10 18:24:01.527157 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:01.527113 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:24:01.527478 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:01.527194 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:24:02.321963 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.321927 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" event={"ID":"08d3e61e-93fc-41cf-93f9-35655cfcbbac","Type":"ContainerStarted","Data":"278f872f48a394b36a72cd3e85b2a07a849312614b193241a92559ff3644c802"} Jul 10 18:24:02.322163 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.322137 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:24:02.324092 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.324041 2571 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-545gn container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 18:24:02.324230 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.324098 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" podUID="08d3e61e-93fc-41cf-93f9-35655cfcbbac" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 18:24:02.324230 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.324180 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-j46xf" event={"ID":"da45c16f-fdef-4fd6-b301-bf1a78e557d7","Type":"ContainerStarted","Data":"e46add446a5f8af807012f5a07d414387592fbe2327727dc10d04a88d00052e2"} Jul 10 18:24:02.324386 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.324339 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-j46xf" Jul 10 18:24:02.326135 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.326111 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-mc5v5" event={"ID":"7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d","Type":"ContainerStarted","Data":"9851ec31619ca19b310a69c01358718ece58ef14acf461cc5a565de4da487d83"} Jul 10 18:24:02.326279 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.326262 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-mc5v5" Jul 10 18:24:02.327973 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.327946 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-ss6xx" event={"ID":"9764f143-44c0-4790-8b60-530bd022797f","Type":"ContainerStarted","Data":"e7aa20b8175d9ca88a9d972213f28961fa86478db8f9391619a7c5eff5880b6d"} Jul 10 18:24:02.345315 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.345259 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" podStartSLOduration=1.618917189 podStartE2EDuration="6.345243459s" totalImagesPullingTime="4.72632627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:23:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:24:02.343060873 +0000 UTC m=+399.242330002" watchObservedRunningTime="2026-07-10 18:24:02.345243459 +0000 UTC m=+399.244512567" Jul 10 18:24:02.361804 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.361724 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-j46xf" podStartSLOduration=2.919938056 podStartE2EDuration="8.361699171s" totalImagesPullingTime="5.441761115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:24:02.361093982 +0000 UTC m=+399.260363112" watchObservedRunningTime="2026-07-10 18:24:02.361699171 +0000 UTC m=+399.260968281" Jul 10 18:24:02.381519 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.381468 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-79bb7c45b8-ss6xx" podStartSLOduration=2.877327063 podStartE2EDuration="8.381447454s" totalImagesPullingTime="5.504120391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:24:02.380968116 +0000 UTC m=+399.280237242" watchObservedRunningTime="2026-07-10 18:24:02.381447454 +0000 UTC m=+399.280716556" Jul 10 18:24:02.405866 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.405790 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-mc5v5" podStartSLOduration=2.806944618 podStartE2EDuration="7.405772418s" totalImagesPullingTime="4.5988278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:24:02.404527147 +0000 UTC m=+399.303796271" watchObservedRunningTime="2026-07-10 18:24:02.405772418 +0000 UTC m=+399.305041525" Jul 10 18:24:02.584884 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.584850 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d"] Jul 10 18:24:02.588183 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.588166 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.592484 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.592467 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-p5k6d\"" Jul 10 18:24:02.609051 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.609019 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d"] Jul 10 18:24:02.667925 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.667894 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw"] Jul 10 18:24:02.671140 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.671118 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.676242 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.676217 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.676397 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.676290 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.676397 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.676325 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-data\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.676397 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.676349 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.676397 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.676388 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4757\" (UniqueName: \"kubernetes.io/projected/e7a7fabc-707d-40e0-b9cd-75814eda3e85-kube-api-access-h4757\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.676577 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.676466 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.676577 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.676518 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.676670 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.676598 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.676706 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.676683 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-token\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.708374 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.708328 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw"] Jul 10 18:24:02.777376 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.777346 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82c47cdc-e129-4ad8-92af-8744507f1872-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.777577 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.777384 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82c47cdc-e129-4ad8-92af-8744507f1872-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.777577 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.777416 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82c47cdc-e129-4ad8-92af-8744507f1872-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.777577 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.777463 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.777577 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.777510 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crgr8\" (UniqueName: \"kubernetes.io/projected/82c47cdc-e129-4ad8-92af-8744507f1872-kube-api-access-crgr8\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.777577 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.777550 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82c47cdc-e129-4ad8-92af-8744507f1872-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.777866 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.777635 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-token\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.777866 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.777669 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c47cdc-e129-4ad8-92af-8744507f1872-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.777866 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.777691 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82c47cdc-e129-4ad8-92af-8744507f1872-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.777866 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.777716 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.777866 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.777787 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.777866 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.777829 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-data\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.777866 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.777851 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.778258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.777886 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82c47cdc-e129-4ad8-92af-8744507f1872-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.778258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.777944 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4757\" (UniqueName: \"kubernetes.io/projected/e7a7fabc-707d-40e0-b9cd-75814eda3e85-kube-api-access-h4757\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.778258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.778015 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.778258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.778067 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c47cdc-e129-4ad8-92af-8744507f1872-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.778258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.778109 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.778258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.778131 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.778258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.778172 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-data\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.778554 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.778349 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.778554 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.778423 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.778554 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.778437 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.780206 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.780173 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.780500 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.780479 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.788397 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.788374 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-token\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.788498 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.788450 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4757\" (UniqueName: \"kubernetes.io/projected/e7a7fabc-707d-40e0-b9cd-75814eda3e85-kube-api-access-h4757\") pod \"data-science-gateway-data-science-gateway-class-548557588-mnk4d\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.879539 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.879453 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82c47cdc-e129-4ad8-92af-8744507f1872-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.879539 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.879495 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82c47cdc-e129-4ad8-92af-8744507f1872-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.879539 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.879518 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82c47cdc-e129-4ad8-92af-8744507f1872-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.879539 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.879539 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crgr8\" (UniqueName: \"kubernetes.io/projected/82c47cdc-e129-4ad8-92af-8744507f1872-kube-api-access-crgr8\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.879884 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.879560 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82c47cdc-e129-4ad8-92af-8744507f1872-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.879884 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.879596 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c47cdc-e129-4ad8-92af-8744507f1872-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.879884 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.879638 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82c47cdc-e129-4ad8-92af-8744507f1872-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.879884 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.879677 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82c47cdc-e129-4ad8-92af-8744507f1872-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.879884 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.879727 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c47cdc-e129-4ad8-92af-8744507f1872-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.880074 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.880007 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82c47cdc-e129-4ad8-92af-8744507f1872-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.880074 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.880029 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82c47cdc-e129-4ad8-92af-8744507f1872-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.880337 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.880298 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c47cdc-e129-4ad8-92af-8744507f1872-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.880337 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.880332 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82c47cdc-e129-4ad8-92af-8744507f1872-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.880479 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.880421 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82c47cdc-e129-4ad8-92af-8744507f1872-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.881979 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.881955 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82c47cdc-e129-4ad8-92af-8744507f1872-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.882095 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.882078 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82c47cdc-e129-4ad8-92af-8744507f1872-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.890274 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.890241 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crgr8\" (UniqueName: \"kubernetes.io/projected/82c47cdc-e129-4ad8-92af-8744507f1872-kube-api-access-crgr8\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.890931 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.890911 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82c47cdc-e129-4ad8-92af-8744507f1872-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678hw6hw\" (UID: \"82c47cdc-e129-4ad8-92af-8744507f1872\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:02.897836 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.897815 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:02.981024 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:02.980976 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:03.035909 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:03.035854 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d"] Jul 10 18:24:03.040052 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:24:03.039997 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7a7fabc_707d_40e0_b9cd_75814eda3e85.slice/crio-3b012e2f6697860b6fcabcd5335456a559e8df474164ae5b15f1698f6e55a830 WatchSource:0}: Error finding container 3b012e2f6697860b6fcabcd5335456a559e8df474164ae5b15f1698f6e55a830: Status 404 returned error can't find the container with id 3b012e2f6697860b6fcabcd5335456a559e8df474164ae5b15f1698f6e55a830 Jul 10 18:24:03.140496 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:03.140469 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw"] Jul 10 18:24:03.140942 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:24:03.140919 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82c47cdc_e129_4ad8_92af_8744507f1872.slice/crio-b6684ab0430d035d3db36f2efceef8d5947607ff3e80009f0d4ef47ed1e164d3 WatchSource:0}: Error finding container b6684ab0430d035d3db36f2efceef8d5947607ff3e80009f0d4ef47ed1e164d3: Status 404 returned error can't find the container with id b6684ab0430d035d3db36f2efceef8d5947607ff3e80009f0d4ef47ed1e164d3 Jul 10 18:24:03.333941 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:03.333908 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" event={"ID":"82c47cdc-e129-4ad8-92af-8744507f1872","Type":"ContainerStarted","Data":"b6684ab0430d035d3db36f2efceef8d5947607ff3e80009f0d4ef47ed1e164d3"} Jul 10 18:24:03.335220 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:03.335182 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" event={"ID":"e7a7fabc-707d-40e0-b9cd-75814eda3e85","Type":"ContainerStarted","Data":"3b012e2f6697860b6fcabcd5335456a559e8df474164ae5b15f1698f6e55a830"} Jul 10 18:24:03.335972 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:03.335948 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:24:05.520112 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.520080 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-m89t9"] Jul 10 18:24:05.528630 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.528590 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-m89t9" Jul 10 18:24:05.534536 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.534511 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 10 18:24:05.534824 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.534801 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-2rtxf\"" Jul 10 18:24:05.538340 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.538229 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6645c"] Jul 10 18:24:05.542970 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.542947 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6645c" Jul 10 18:24:05.551960 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.551938 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 10 18:24:05.552976 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.552959 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-8dq48\"" Jul 10 18:24:05.566818 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.566782 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-m89t9"] Jul 10 18:24:05.587528 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.587497 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6645c"] Jul 10 18:24:05.603533 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.603499 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c5eed36-9d1c-487a-98f7-a01d4d540b8c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6645c\" (UID: \"1c5eed36-9d1c-487a-98f7-a01d4d540b8c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6645c" Jul 10 18:24:05.603716 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.603552 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvvvz\" (UniqueName: \"kubernetes.io/projected/147ffda5-ecb7-4e1e-adf7-bb36bdbe42ee-kube-api-access-dvvvz\") pod \"kserve-controller-manager-d6f4b65b8-m89t9\" (UID: \"147ffda5-ecb7-4e1e-adf7-bb36bdbe42ee\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-m89t9" Jul 10 18:24:05.603716 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.603607 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/147ffda5-ecb7-4e1e-adf7-bb36bdbe42ee-cert\") pod \"kserve-controller-manager-d6f4b65b8-m89t9\" (UID: \"147ffda5-ecb7-4e1e-adf7-bb36bdbe42ee\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-m89t9" Jul 10 18:24:05.603857 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.603727 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs7bx\" (UniqueName: \"kubernetes.io/projected/1c5eed36-9d1c-487a-98f7-a01d4d540b8c-kube-api-access-fs7bx\") pod \"llmisvc-controller-manager-777c6bd7d5-6645c\" (UID: \"1c5eed36-9d1c-487a-98f7-a01d4d540b8c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6645c" Jul 10 18:24:05.705157 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.705126 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c5eed36-9d1c-487a-98f7-a01d4d540b8c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6645c\" (UID: \"1c5eed36-9d1c-487a-98f7-a01d4d540b8c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6645c" Jul 10 18:24:05.705333 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.705172 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvvvz\" (UniqueName: \"kubernetes.io/projected/147ffda5-ecb7-4e1e-adf7-bb36bdbe42ee-kube-api-access-dvvvz\") pod \"kserve-controller-manager-d6f4b65b8-m89t9\" (UID: \"147ffda5-ecb7-4e1e-adf7-bb36bdbe42ee\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-m89t9" Jul 10 18:24:05.705333 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:24:05.705265 2571 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 10 18:24:05.705333 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.705317 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/147ffda5-ecb7-4e1e-adf7-bb36bdbe42ee-cert\") pod \"kserve-controller-manager-d6f4b65b8-m89t9\" (UID: \"147ffda5-ecb7-4e1e-adf7-bb36bdbe42ee\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-m89t9" Jul 10 18:24:05.705333 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:24:05.705326 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c5eed36-9d1c-487a-98f7-a01d4d540b8c-cert podName:1c5eed36-9d1c-487a-98f7-a01d4d540b8c nodeName:}" failed. No retries permitted until 2026-07-10 18:24:06.20530855 +0000 UTC m=+403.104577636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1c5eed36-9d1c-487a-98f7-a01d4d540b8c-cert") pod "llmisvc-controller-manager-777c6bd7d5-6645c" (UID: "1c5eed36-9d1c-487a-98f7-a01d4d540b8c") : secret "llmisvc-webhook-server-cert" not found Jul 10 18:24:05.705481 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.705398 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs7bx\" (UniqueName: \"kubernetes.io/projected/1c5eed36-9d1c-487a-98f7-a01d4d540b8c-kube-api-access-fs7bx\") pod \"llmisvc-controller-manager-777c6bd7d5-6645c\" (UID: \"1c5eed36-9d1c-487a-98f7-a01d4d540b8c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6645c" Jul 10 18:24:05.707834 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.707812 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/147ffda5-ecb7-4e1e-adf7-bb36bdbe42ee-cert\") pod \"kserve-controller-manager-d6f4b65b8-m89t9\" (UID: \"147ffda5-ecb7-4e1e-adf7-bb36bdbe42ee\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-m89t9" Jul 10 18:24:05.719114 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.719087 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs7bx\" (UniqueName: \"kubernetes.io/projected/1c5eed36-9d1c-487a-98f7-a01d4d540b8c-kube-api-access-fs7bx\") pod \"llmisvc-controller-manager-777c6bd7d5-6645c\" (UID: \"1c5eed36-9d1c-487a-98f7-a01d4d540b8c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6645c" Jul 10 18:24:05.719238 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.719153 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvvvz\" (UniqueName: \"kubernetes.io/projected/147ffda5-ecb7-4e1e-adf7-bb36bdbe42ee-kube-api-access-dvvvz\") pod \"kserve-controller-manager-d6f4b65b8-m89t9\" (UID: \"147ffda5-ecb7-4e1e-adf7-bb36bdbe42ee\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-m89t9" Jul 10 18:24:05.737321 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.737284 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:24:05.737448 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.737369 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:24:05.737448 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.737410 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:24:05.742908 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.742878 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:24:05.742989 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.742950 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:24:05.742989 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.742979 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:24:05.842123 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:05.841863 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-m89t9" Jul 10 18:24:06.031123 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:06.031094 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-m89t9"] Jul 10 18:24:06.033921 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:24:06.033896 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod147ffda5_ecb7_4e1e_adf7_bb36bdbe42ee.slice/crio-7871ecc76d471d87b1107ac47c6370ee58e01f65c68b3a547408db85e3880fe1 WatchSource:0}: Error finding container 7871ecc76d471d87b1107ac47c6370ee58e01f65c68b3a547408db85e3880fe1: Status 404 returned error can't find the container with id 7871ecc76d471d87b1107ac47c6370ee58e01f65c68b3a547408db85e3880fe1 Jul 10 18:24:06.209005 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:06.208961 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c5eed36-9d1c-487a-98f7-a01d4d540b8c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6645c\" (UID: \"1c5eed36-9d1c-487a-98f7-a01d4d540b8c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6645c" Jul 10 18:24:06.211510 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:06.211483 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c5eed36-9d1c-487a-98f7-a01d4d540b8c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6645c\" (UID: \"1c5eed36-9d1c-487a-98f7-a01d4d540b8c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6645c" Jul 10 18:24:06.352656 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:06.352536 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-m89t9" event={"ID":"147ffda5-ecb7-4e1e-adf7-bb36bdbe42ee","Type":"ContainerStarted","Data":"7871ecc76d471d87b1107ac47c6370ee58e01f65c68b3a547408db85e3880fe1"} Jul 10 18:24:06.354057 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:06.354024 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" event={"ID":"82c47cdc-e129-4ad8-92af-8744507f1872","Type":"ContainerStarted","Data":"69689b7ade2fe0c936a657d73f5d2985197b73b8d087cdf706234d89950b346a"} Jul 10 18:24:06.355511 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:06.355487 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" event={"ID":"e7a7fabc-707d-40e0-b9cd-75814eda3e85","Type":"ContainerStarted","Data":"239e6cf25ab93c2a8666e184fea7568295b0d1815f9e374ddac5e1f2e80d8d9a"} Jul 10 18:24:06.385483 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:06.385438 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" podStartSLOduration=1.79144544 podStartE2EDuration="4.385419521s" totalImagesPullingTime="2.593974081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:24:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:24:06.382878107 +0000 UTC m=+403.282147216" watchObservedRunningTime="2026-07-10 18:24:06.385419521 +0000 UTC m=+403.284688706" Jul 10 18:24:06.415779 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:06.415719 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" podStartSLOduration=1.716431177 podStartE2EDuration="4.415696258s" totalImagesPullingTime="2.699265081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:24:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:24:06.407934807 +0000 UTC m=+403.307203918" watchObservedRunningTime="2026-07-10 18:24:06.415696258 +0000 UTC m=+403.314965366" Jul 10 18:24:06.453952 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:06.453909 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6645c" Jul 10 18:24:06.605964 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:06.605874 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6645c"] Jul 10 18:24:06.621772 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:24:06.621747 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1c5eed36_9d1c_487a_98f7_a01d4d540b8c.slice/crio-ec389b1555dca3ff4b1b74a8b15d34c1834a3b67b0905a0eac78f1e4bb421a49 WatchSource:0}: Error finding container ec389b1555dca3ff4b1b74a8b15d34c1834a3b67b0905a0eac78f1e4bb421a49: Status 404 returned error can't find the container with id ec389b1555dca3ff4b1b74a8b15d34c1834a3b67b0905a0eac78f1e4bb421a49 Jul 10 18:24:06.898839 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:06.898742 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:06.900715 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:06.900672 2571 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-548557588-mnk4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused" start-of-body= Jul 10 18:24:06.900845 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:06.900744 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" podUID="e7a7fabc-707d-40e0-b9cd-75814eda3e85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused" Jul 10 18:24:06.982299 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:06.982254 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:06.988145 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:06.988108 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:07.362769 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:07.362733 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6645c" event={"ID":"1c5eed36-9d1c-487a-98f7-a01d4d540b8c","Type":"ContainerStarted","Data":"ec389b1555dca3ff4b1b74a8b15d34c1834a3b67b0905a0eac78f1e4bb421a49"} Jul 10 18:24:07.363212 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:07.363148 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:07.364519 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:07.364494 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678hw6hw" Jul 10 18:24:07.429091 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:07.429061 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d"] Jul 10 18:24:07.898448 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:07.898397 2571 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-548557588-mnk4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused" start-of-body= Jul 10 18:24:07.898922 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:07.898477 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" podUID="e7a7fabc-707d-40e0-b9cd-75814eda3e85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused" Jul 10 18:24:08.898561 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:08.898518 2571 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-548557588-mnk4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused" start-of-body= Jul 10 18:24:08.898975 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:08.898574 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" podUID="e7a7fabc-707d-40e0-b9cd-75814eda3e85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.20:15021/healthz/ready\": dial tcp 10.134.0.20:15021: connect: connection refused" Jul 10 18:24:08.976822 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:08.976778 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-545gn"] Jul 10 18:24:08.977052 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:08.977020 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" podUID="08d3e61e-93fc-41cf-93f9-35655cfcbbac" containerName="discovery" containerID="cri-o://278f872f48a394b36a72cd3e85b2a07a849312614b193241a92559ff3644c802" gracePeriod=30 Jul 10 18:24:09.238340 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.238315 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:24:09.340102 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.340063 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9bmd\" (UniqueName: \"kubernetes.io/projected/08d3e61e-93fc-41cf-93f9-35655cfcbbac-kube-api-access-x9bmd\") pod \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " Jul 10 18:24:09.340309 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.340144 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/08d3e61e-93fc-41cf-93f9-35655cfcbbac-local-certs\") pod \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " Jul 10 18:24:09.340309 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.340197 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-csr-dns-cert\") pod \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " Jul 10 18:24:09.340309 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.340224 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-csr-ca-configmap\") pod \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " Jul 10 18:24:09.340309 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.340260 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/08d3e61e-93fc-41cf-93f9-35655cfcbbac-cacerts\") pod \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " Jul 10 18:24:09.340309 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.340287 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-kubeconfig\") pod \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " Jul 10 18:24:09.340573 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.340330 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-token\") pod \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\" (UID: \"08d3e61e-93fc-41cf-93f9-35655cfcbbac\") " Jul 10 18:24:09.341250 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.341194 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "08d3e61e-93fc-41cf-93f9-35655cfcbbac" (UID: "08d3e61e-93fc-41cf-93f9-35655cfcbbac"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:24:09.343359 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.343304 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-token" (OuterVolumeSpecName: "istio-token") pod "08d3e61e-93fc-41cf-93f9-35655cfcbbac" (UID: "08d3e61e-93fc-41cf-93f9-35655cfcbbac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:24:09.343359 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.343326 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "08d3e61e-93fc-41cf-93f9-35655cfcbbac" (UID: "08d3e61e-93fc-41cf-93f9-35655cfcbbac"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:24:09.343557 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.343413 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08d3e61e-93fc-41cf-93f9-35655cfcbbac-cacerts" (OuterVolumeSpecName: "cacerts") pod "08d3e61e-93fc-41cf-93f9-35655cfcbbac" (UID: "08d3e61e-93fc-41cf-93f9-35655cfcbbac"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:24:09.343669 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.343641 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "08d3e61e-93fc-41cf-93f9-35655cfcbbac" (UID: "08d3e61e-93fc-41cf-93f9-35655cfcbbac"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:24:09.343778 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.343669 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08d3e61e-93fc-41cf-93f9-35655cfcbbac-local-certs" (OuterVolumeSpecName: "local-certs") pod "08d3e61e-93fc-41cf-93f9-35655cfcbbac" (UID: "08d3e61e-93fc-41cf-93f9-35655cfcbbac"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:24:09.343833 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.343796 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08d3e61e-93fc-41cf-93f9-35655cfcbbac-kube-api-access-x9bmd" (OuterVolumeSpecName: "kube-api-access-x9bmd") pod "08d3e61e-93fc-41cf-93f9-35655cfcbbac" (UID: "08d3e61e-93fc-41cf-93f9-35655cfcbbac"). InnerVolumeSpecName "kube-api-access-x9bmd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:24:09.375741 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.375702 2571 generic.go:356] "Generic (PLEG): container finished" podID="08d3e61e-93fc-41cf-93f9-35655cfcbbac" containerID="278f872f48a394b36a72cd3e85b2a07a849312614b193241a92559ff3644c802" exitCode=0 Jul 10 18:24:09.375912 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.375761 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" Jul 10 18:24:09.375912 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.375864 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" event={"ID":"08d3e61e-93fc-41cf-93f9-35655cfcbbac","Type":"ContainerDied","Data":"278f872f48a394b36a72cd3e85b2a07a849312614b193241a92559ff3644c802"} Jul 10 18:24:09.375912 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.375911 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-545gn" event={"ID":"08d3e61e-93fc-41cf-93f9-35655cfcbbac","Type":"ContainerDied","Data":"e6028ec81db536e91cd02e79ecd8842314fa6a402884d5bf409ca08ed61c81c9"} Jul 10 18:24:09.376094 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.375936 2571 scope.go:117] "RemoveContainer" containerID="278f872f48a394b36a72cd3e85b2a07a849312614b193241a92559ff3644c802" Jul 10 18:24:09.377741 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.377679 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-m89t9" event={"ID":"147ffda5-ecb7-4e1e-adf7-bb36bdbe42ee","Type":"ContainerStarted","Data":"82c9b97f73d4ddf190d7c83f71d82146d5b81e69e4da1c680a012a7291a4c594"} Jul 10 18:24:09.377858 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.377785 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" podUID="e7a7fabc-707d-40e0-b9cd-75814eda3e85" containerName="istio-proxy" containerID="cri-o://239e6cf25ab93c2a8666e184fea7568295b0d1815f9e374ddac5e1f2e80d8d9a" gracePeriod=30 Jul 10 18:24:09.400078 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.400025 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-m89t9" podStartSLOduration=1.648799806 podStartE2EDuration="4.40000569s" totalImagesPullingTime="2.751205884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:24:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:24:09.399294355 +0000 UTC m=+406.298563465" watchObservedRunningTime="2026-07-10 18:24:09.40000569 +0000 UTC m=+406.299274798" Jul 10 18:24:09.442076 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.442038 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-token\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:24:09.442076 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.442072 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9bmd\" (UniqueName: \"kubernetes.io/projected/08d3e61e-93fc-41cf-93f9-35655cfcbbac-kube-api-access-x9bmd\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:24:09.442076 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.442085 2571 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/08d3e61e-93fc-41cf-93f9-35655cfcbbac-local-certs\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:24:09.442321 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.442098 2571 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-csr-dns-cert\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:24:09.442321 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.442112 2571 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-csr-ca-configmap\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:24:09.442321 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.442124 2571 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/08d3e61e-93fc-41cf-93f9-35655cfcbbac-cacerts\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:24:09.442321 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.442138 2571 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/08d3e61e-93fc-41cf-93f9-35655cfcbbac-istio-kubeconfig\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:24:09.445056 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.445029 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-545gn"] Jul 10 18:24:09.453275 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.452638 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-545gn"] Jul 10 18:24:09.685239 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.685136 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08d3e61e-93fc-41cf-93f9-35655cfcbbac" path="/var/lib/kubelet/pods/08d3e61e-93fc-41cf-93f9-35655cfcbbac/volumes" Jul 10 18:24:09.784941 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.784913 2571 scope.go:117] "RemoveContainer" containerID="278f872f48a394b36a72cd3e85b2a07a849312614b193241a92559ff3644c802" Jul 10 18:24:09.785275 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:24:09.785252 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"278f872f48a394b36a72cd3e85b2a07a849312614b193241a92559ff3644c802\": container with ID starting with 278f872f48a394b36a72cd3e85b2a07a849312614b193241a92559ff3644c802 not found: ID does not exist" containerID="278f872f48a394b36a72cd3e85b2a07a849312614b193241a92559ff3644c802" Jul 10 18:24:09.785351 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:09.785290 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"278f872f48a394b36a72cd3e85b2a07a849312614b193241a92559ff3644c802"} err="failed to get container status \"278f872f48a394b36a72cd3e85b2a07a849312614b193241a92559ff3644c802\": rpc error: code = NotFound desc = could not find container \"278f872f48a394b36a72cd3e85b2a07a849312614b193241a92559ff3644c802\": container with ID starting with 278f872f48a394b36a72cd3e85b2a07a849312614b193241a92559ff3644c802 not found: ID does not exist" Jul 10 18:24:10.384921 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:10.384881 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6645c" event={"ID":"1c5eed36-9d1c-487a-98f7-a01d4d540b8c","Type":"ContainerStarted","Data":"1ac08013cee3f731e1a041c6e2a4c5da99897f60b7ae3e19b81f49e4d64b4310"} Jul 10 18:24:10.385417 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:10.385065 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6645c" Jul 10 18:24:10.385417 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:10.385104 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-m89t9" Jul 10 18:24:10.402830 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:10.402777 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6645c" podStartSLOduration=2.192644261 podStartE2EDuration="5.402759653s" totalImagesPullingTime="3.210115392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:24:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:24:10.400605657 +0000 UTC m=+407.299874881" watchObservedRunningTime="2026-07-10 18:24:10.402759653 +0000 UTC m=+407.302028773" Jul 10 18:24:13.337908 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:13.337877 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-j46xf" Jul 10 18:24:13.339942 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:13.339920 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-mc5v5" Jul 10 18:24:14.626115 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.626086 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:14.791189 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.791157 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-token\") pod \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " Jul 10 18:24:14.791380 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.791203 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-podinfo\") pod \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " Jul 10 18:24:14.791380 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.791252 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-workload-socket\") pod \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " Jul 10 18:24:14.791380 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.791272 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-envoy\") pod \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " Jul 10 18:24:14.791380 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.791303 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-data\") pod \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " Jul 10 18:24:14.791380 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.791329 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-credential-socket\") pod \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " Jul 10 18:24:14.791380 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.791359 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istiod-ca-cert\") pod \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " Jul 10 18:24:14.791723 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.791395 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4757\" (UniqueName: \"kubernetes.io/projected/e7a7fabc-707d-40e0-b9cd-75814eda3e85-kube-api-access-h4757\") pod \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " Jul 10 18:24:14.791723 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.791452 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-workload-certs\") pod \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\" (UID: \"e7a7fabc-707d-40e0-b9cd-75814eda3e85\") " Jul 10 18:24:14.791723 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.791584 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e7a7fabc-707d-40e0-b9cd-75814eda3e85" (UID: "e7a7fabc-707d-40e0-b9cd-75814eda3e85"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:24:14.791723 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.791643 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-data" (OuterVolumeSpecName: "istio-data") pod "e7a7fabc-707d-40e0-b9cd-75814eda3e85" (UID: "e7a7fabc-707d-40e0-b9cd-75814eda3e85"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:24:14.791723 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.791652 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e7a7fabc-707d-40e0-b9cd-75814eda3e85" (UID: "e7a7fabc-707d-40e0-b9cd-75814eda3e85"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:24:14.792034 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.791737 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-credential-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:24:14.792034 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.791838 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e7a7fabc-707d-40e0-b9cd-75814eda3e85" (UID: "e7a7fabc-707d-40e0-b9cd-75814eda3e85"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:24:14.792034 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.791871 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e7a7fabc-707d-40e0-b9cd-75814eda3e85" (UID: "e7a7fabc-707d-40e0-b9cd-75814eda3e85"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:24:14.793527 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.793498 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e7a7fabc-707d-40e0-b9cd-75814eda3e85" (UID: "e7a7fabc-707d-40e0-b9cd-75814eda3e85"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:24:14.793679 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.793564 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7a7fabc-707d-40e0-b9cd-75814eda3e85-kube-api-access-h4757" (OuterVolumeSpecName: "kube-api-access-h4757") pod "e7a7fabc-707d-40e0-b9cd-75814eda3e85" (UID: "e7a7fabc-707d-40e0-b9cd-75814eda3e85"). InnerVolumeSpecName "kube-api-access-h4757". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:24:14.793679 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.793567 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-token" (OuterVolumeSpecName: "istio-token") pod "e7a7fabc-707d-40e0-b9cd-75814eda3e85" (UID: "e7a7fabc-707d-40e0-b9cd-75814eda3e85"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:24:14.793679 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.793564 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e7a7fabc-707d-40e0-b9cd-75814eda3e85" (UID: "e7a7fabc-707d-40e0-b9cd-75814eda3e85"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 18:24:14.892803 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.892762 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-podinfo\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:24:14.892803 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.892798 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-workload-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:24:14.892803 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.892809 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-envoy\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:24:14.893050 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.892821 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-data\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:24:14.893050 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.892833 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istiod-ca-cert\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:24:14.893050 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.892844 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4757\" (UniqueName: \"kubernetes.io/projected/e7a7fabc-707d-40e0-b9cd-75814eda3e85-kube-api-access-h4757\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:24:14.893050 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.892855 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e7a7fabc-707d-40e0-b9cd-75814eda3e85-workload-certs\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:24:14.893050 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:14.892865 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e7a7fabc-707d-40e0-b9cd-75814eda3e85-istio-token\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:24:15.408549 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:15.408514 2571 generic.go:356] "Generic (PLEG): container finished" podID="e7a7fabc-707d-40e0-b9cd-75814eda3e85" containerID="239e6cf25ab93c2a8666e184fea7568295b0d1815f9e374ddac5e1f2e80d8d9a" exitCode=0 Jul 10 18:24:15.408737 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:15.408579 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" Jul 10 18:24:15.408737 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:15.408599 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" event={"ID":"e7a7fabc-707d-40e0-b9cd-75814eda3e85","Type":"ContainerDied","Data":"239e6cf25ab93c2a8666e184fea7568295b0d1815f9e374ddac5e1f2e80d8d9a"} Jul 10 18:24:15.408737 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:15.408673 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d" event={"ID":"e7a7fabc-707d-40e0-b9cd-75814eda3e85","Type":"ContainerDied","Data":"3b012e2f6697860b6fcabcd5335456a559e8df474164ae5b15f1698f6e55a830"} Jul 10 18:24:15.408737 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:15.408696 2571 scope.go:117] "RemoveContainer" containerID="239e6cf25ab93c2a8666e184fea7568295b0d1815f9e374ddac5e1f2e80d8d9a" Jul 10 18:24:15.417848 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:15.417831 2571 scope.go:117] "RemoveContainer" containerID="239e6cf25ab93c2a8666e184fea7568295b0d1815f9e374ddac5e1f2e80d8d9a" Jul 10 18:24:15.418152 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:24:15.418132 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"239e6cf25ab93c2a8666e184fea7568295b0d1815f9e374ddac5e1f2e80d8d9a\": container with ID starting with 239e6cf25ab93c2a8666e184fea7568295b0d1815f9e374ddac5e1f2e80d8d9a not found: ID does not exist" containerID="239e6cf25ab93c2a8666e184fea7568295b0d1815f9e374ddac5e1f2e80d8d9a" Jul 10 18:24:15.418223 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:15.418159 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"239e6cf25ab93c2a8666e184fea7568295b0d1815f9e374ddac5e1f2e80d8d9a"} err="failed to get container status \"239e6cf25ab93c2a8666e184fea7568295b0d1815f9e374ddac5e1f2e80d8d9a\": rpc error: code = NotFound desc = could not find container \"239e6cf25ab93c2a8666e184fea7568295b0d1815f9e374ddac5e1f2e80d8d9a\": container with ID starting with 239e6cf25ab93c2a8666e184fea7568295b0d1815f9e374ddac5e1f2e80d8d9a not found: ID does not exist" Jul 10 18:24:15.429718 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:15.429685 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d"] Jul 10 18:24:15.434417 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:15.434392 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-mnk4d"] Jul 10 18:24:15.683500 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:15.683416 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7a7fabc-707d-40e0-b9cd-75814eda3e85" path="/var/lib/kubelet/pods/e7a7fabc-707d-40e0-b9cd-75814eda3e85/volumes" Jul 10 18:24:41.391594 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:41.391562 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6645c" Jul 10 18:24:41.394723 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:41.394704 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-m89t9" Jul 10 18:24:55.542531 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:55.542418 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qkhwx"] Jul 10 18:24:55.543095 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:55.542853 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7a7fabc-707d-40e0-b9cd-75814eda3e85" containerName="istio-proxy" Jul 10 18:24:55.543095 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:55.542872 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7a7fabc-707d-40e0-b9cd-75814eda3e85" containerName="istio-proxy" Jul 10 18:24:55.543095 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:55.542899 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08d3e61e-93fc-41cf-93f9-35655cfcbbac" containerName="discovery" Jul 10 18:24:55.543095 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:55.542907 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="08d3e61e-93fc-41cf-93f9-35655cfcbbac" containerName="discovery" Jul 10 18:24:55.543095 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:55.542992 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7a7fabc-707d-40e0-b9cd-75814eda3e85" containerName="istio-proxy" Jul 10 18:24:55.543095 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:55.543006 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="08d3e61e-93fc-41cf-93f9-35655cfcbbac" containerName="discovery" Jul 10 18:24:55.545262 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:55.545240 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qkhwx" Jul 10 18:24:55.549281 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:55.549253 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 18:24:55.549442 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:55.549312 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 18:24:55.549442 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:55.549345 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-96c9g\"" Jul 10 18:24:55.558387 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:55.558353 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qkhwx"] Jul 10 18:24:55.630797 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:55.630762 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz28k\" (UniqueName: \"kubernetes.io/projected/8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae-kube-api-access-mz28k\") pod \"kuadrant-operator-catalog-qkhwx\" (UID: \"8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae\") " pod="kuadrant-system/kuadrant-operator-catalog-qkhwx" Jul 10 18:24:55.731554 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:55.731519 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz28k\" (UniqueName: \"kubernetes.io/projected/8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae-kube-api-access-mz28k\") pod \"kuadrant-operator-catalog-qkhwx\" (UID: \"8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae\") " pod="kuadrant-system/kuadrant-operator-catalog-qkhwx" Jul 10 18:24:55.739843 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:55.739809 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz28k\" (UniqueName: \"kubernetes.io/projected/8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae-kube-api-access-mz28k\") pod \"kuadrant-operator-catalog-qkhwx\" (UID: \"8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae\") " pod="kuadrant-system/kuadrant-operator-catalog-qkhwx" Jul 10 18:24:55.855550 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:55.855444 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qkhwx" Jul 10 18:24:55.907514 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:55.907475 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qkhwx"] Jul 10 18:24:55.999523 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:55.999500 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qkhwx"] Jul 10 18:24:56.002275 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:24:56.002243 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c4f59f0_a0bf_4af0_bf46_ad2bbe64e0ae.slice/crio-c7cb5df781949bedc6e523791ea900c67c22302506ed408b99a23f3ca7382401 WatchSource:0}: Error finding container c7cb5df781949bedc6e523791ea900c67c22302506ed408b99a23f3ca7382401: Status 404 returned error can't find the container with id c7cb5df781949bedc6e523791ea900c67c22302506ed408b99a23f3ca7382401 Jul 10 18:24:56.118029 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:56.117948 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kqcg5"] Jul 10 18:24:56.120879 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:56.120862 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kqcg5" Jul 10 18:24:56.128805 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:56.128780 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kqcg5"] Jul 10 18:24:56.235249 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:56.235210 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cmtw\" (UniqueName: \"kubernetes.io/projected/e1f1a3b1-0c3a-4ea5-8370-70a73e7904f7-kube-api-access-7cmtw\") pod \"kuadrant-operator-catalog-kqcg5\" (UID: \"e1f1a3b1-0c3a-4ea5-8370-70a73e7904f7\") " pod="kuadrant-system/kuadrant-operator-catalog-kqcg5" Jul 10 18:24:56.336138 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:56.336096 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cmtw\" (UniqueName: \"kubernetes.io/projected/e1f1a3b1-0c3a-4ea5-8370-70a73e7904f7-kube-api-access-7cmtw\") pod \"kuadrant-operator-catalog-kqcg5\" (UID: \"e1f1a3b1-0c3a-4ea5-8370-70a73e7904f7\") " pod="kuadrant-system/kuadrant-operator-catalog-kqcg5" Jul 10 18:24:56.344168 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:56.344137 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cmtw\" (UniqueName: \"kubernetes.io/projected/e1f1a3b1-0c3a-4ea5-8370-70a73e7904f7-kube-api-access-7cmtw\") pod \"kuadrant-operator-catalog-kqcg5\" (UID: \"e1f1a3b1-0c3a-4ea5-8370-70a73e7904f7\") " pod="kuadrant-system/kuadrant-operator-catalog-kqcg5" Jul 10 18:24:56.430389 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:56.430295 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kqcg5" Jul 10 18:24:56.576254 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:56.576227 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kqcg5"] Jul 10 18:24:56.578825 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:24:56.578795 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1f1a3b1_0c3a_4ea5_8370_70a73e7904f7.slice/crio-95612bd47df6306d4a055a1ad861dfe3dfc8d9206706043efcd597847a210aa0 WatchSource:0}: Error finding container 95612bd47df6306d4a055a1ad861dfe3dfc8d9206706043efcd597847a210aa0: Status 404 returned error can't find the container with id 95612bd47df6306d4a055a1ad861dfe3dfc8d9206706043efcd597847a210aa0 Jul 10 18:24:56.580339 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:56.580312 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qkhwx" event={"ID":"8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae","Type":"ContainerStarted","Data":"c7cb5df781949bedc6e523791ea900c67c22302506ed408b99a23f3ca7382401"} Jul 10 18:24:57.587710 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:57.587661 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kqcg5" event={"ID":"e1f1a3b1-0c3a-4ea5-8370-70a73e7904f7","Type":"ContainerStarted","Data":"95612bd47df6306d4a055a1ad861dfe3dfc8d9206706043efcd597847a210aa0"} Jul 10 18:24:58.595198 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:58.595156 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qkhwx" event={"ID":"8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae","Type":"ContainerStarted","Data":"9c568fb43e62e3dfb9e32be3a3e38888fb20db7effe78c3575c0e2083ddac1d4"} Jul 10 18:24:58.595699 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:58.595200 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-qkhwx" podUID="8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae" containerName="registry-server" containerID="cri-o://9c568fb43e62e3dfb9e32be3a3e38888fb20db7effe78c3575c0e2083ddac1d4" gracePeriod=2 Jul 10 18:24:58.596749 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:58.596726 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kqcg5" event={"ID":"e1f1a3b1-0c3a-4ea5-8370-70a73e7904f7","Type":"ContainerStarted","Data":"bd77ce2ce98cf2fcaf9c2eb7ffec8a48edb0ec71dadde6f8bf34854c52a83bb8"} Jul 10 18:24:58.611035 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:58.610992 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-qkhwx" podStartSLOduration=1.576657454 podStartE2EDuration="3.610975689s" totalImagesPullingTime="2.034318235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:24:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:24:58.609797453 +0000 UTC m=+455.509066590" watchObservedRunningTime="2026-07-10 18:24:58.610975689 +0000 UTC m=+455.510244817" Jul 10 18:24:58.625219 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:58.625170 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-kqcg5" podStartSLOduration=1.16360622 podStartE2EDuration="2.625151743s" totalImagesPullingTime="1.461545523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:24:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:24:58.624863713 +0000 UTC m=+455.524132848" watchObservedRunningTime="2026-07-10 18:24:58.625151743 +0000 UTC m=+455.524420850" Jul 10 18:24:58.839851 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:58.839829 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qkhwx" Jul 10 18:24:58.958015 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:58.957927 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mz28k\" (UniqueName: \"kubernetes.io/projected/8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae-kube-api-access-mz28k\") pod \"8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae\" (UID: \"8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae\") " Jul 10 18:24:58.960080 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:58.960045 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae-kube-api-access-mz28k" (OuterVolumeSpecName: "kube-api-access-mz28k") pod "8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae" (UID: "8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae"). InnerVolumeSpecName "kube-api-access-mz28k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:24:59.058839 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:59.058811 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mz28k\" (UniqueName: \"kubernetes.io/projected/8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae-kube-api-access-mz28k\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:24:59.603227 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:59.603188 2571 generic.go:356] "Generic (PLEG): container finished" podID="8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae" containerID="9c568fb43e62e3dfb9e32be3a3e38888fb20db7effe78c3575c0e2083ddac1d4" exitCode=0 Jul 10 18:24:59.603737 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:59.603237 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qkhwx" Jul 10 18:24:59.603737 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:59.603282 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qkhwx" event={"ID":"8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae","Type":"ContainerDied","Data":"9c568fb43e62e3dfb9e32be3a3e38888fb20db7effe78c3575c0e2083ddac1d4"} Jul 10 18:24:59.603737 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:59.603330 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qkhwx" event={"ID":"8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae","Type":"ContainerDied","Data":"c7cb5df781949bedc6e523791ea900c67c22302506ed408b99a23f3ca7382401"} Jul 10 18:24:59.603737 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:59.603350 2571 scope.go:117] "RemoveContainer" containerID="9c568fb43e62e3dfb9e32be3a3e38888fb20db7effe78c3575c0e2083ddac1d4" Jul 10 18:24:59.612413 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:59.612394 2571 scope.go:117] "RemoveContainer" containerID="9c568fb43e62e3dfb9e32be3a3e38888fb20db7effe78c3575c0e2083ddac1d4" Jul 10 18:24:59.612879 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:24:59.612858 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c568fb43e62e3dfb9e32be3a3e38888fb20db7effe78c3575c0e2083ddac1d4\": container with ID starting with 9c568fb43e62e3dfb9e32be3a3e38888fb20db7effe78c3575c0e2083ddac1d4 not found: ID does not exist" containerID="9c568fb43e62e3dfb9e32be3a3e38888fb20db7effe78c3575c0e2083ddac1d4" Jul 10 18:24:59.612940 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:59.612889 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c568fb43e62e3dfb9e32be3a3e38888fb20db7effe78c3575c0e2083ddac1d4"} err="failed to get container status \"9c568fb43e62e3dfb9e32be3a3e38888fb20db7effe78c3575c0e2083ddac1d4\": rpc error: code = NotFound desc = could not find container \"9c568fb43e62e3dfb9e32be3a3e38888fb20db7effe78c3575c0e2083ddac1d4\": container with ID starting with 9c568fb43e62e3dfb9e32be3a3e38888fb20db7effe78c3575c0e2083ddac1d4 not found: ID does not exist" Jul 10 18:24:59.624743 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:59.624715 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qkhwx"] Jul 10 18:24:59.626724 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:59.626704 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qkhwx"] Jul 10 18:24:59.683531 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:24:59.683497 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae" path="/var/lib/kubelet/pods/8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae/volumes" Jul 10 18:25:06.430979 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:06.430939 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-kqcg5" Jul 10 18:25:06.431452 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:06.431009 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-kqcg5" Jul 10 18:25:06.454391 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:06.454364 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-kqcg5" Jul 10 18:25:06.654448 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:06.654416 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-kqcg5" Jul 10 18:25:14.208713 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.208680 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf"] Jul 10 18:25:14.209114 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.208993 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae" containerName="registry-server" Jul 10 18:25:14.209114 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.209004 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae" containerName="registry-server" Jul 10 18:25:14.209114 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.209057 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c4f59f0-a0bf-4af0-bf46-ad2bbe64e0ae" containerName="registry-server" Jul 10 18:25:14.211964 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.211946 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf" Jul 10 18:25:14.214778 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.214629 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-c6frz\"" Jul 10 18:25:14.221372 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.221346 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf"] Jul 10 18:25:14.281920 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.281881 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f63cc50-f78f-4814-b47d-93b0efefafa1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf\" (UID: \"3f63cc50-f78f-4814-b47d-93b0efefafa1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf" Jul 10 18:25:14.282108 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.281957 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f63cc50-f78f-4814-b47d-93b0efefafa1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf\" (UID: \"3f63cc50-f78f-4814-b47d-93b0efefafa1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf" Jul 10 18:25:14.282108 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.281978 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drj46\" (UniqueName: \"kubernetes.io/projected/3f63cc50-f78f-4814-b47d-93b0efefafa1-kube-api-access-drj46\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf\" (UID: \"3f63cc50-f78f-4814-b47d-93b0efefafa1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf" Jul 10 18:25:14.306886 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.306850 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6"] Jul 10 18:25:14.309134 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.309119 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6" Jul 10 18:25:14.317758 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.317725 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6"] Jul 10 18:25:14.382385 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.382341 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f63cc50-f78f-4814-b47d-93b0efefafa1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf\" (UID: \"3f63cc50-f78f-4814-b47d-93b0efefafa1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf" Jul 10 18:25:14.382580 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.382414 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7064372d-282f-48d8-a557-e4a5111bca35-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6\" (UID: \"7064372d-282f-48d8-a557-e4a5111bca35\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6" Jul 10 18:25:14.382580 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.382456 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lwtl\" (UniqueName: \"kubernetes.io/projected/7064372d-282f-48d8-a557-e4a5111bca35-kube-api-access-5lwtl\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6\" (UID: \"7064372d-282f-48d8-a557-e4a5111bca35\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6" Jul 10 18:25:14.382580 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.382484 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7064372d-282f-48d8-a557-e4a5111bca35-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6\" (UID: \"7064372d-282f-48d8-a557-e4a5111bca35\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6" Jul 10 18:25:14.382580 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.382548 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f63cc50-f78f-4814-b47d-93b0efefafa1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf\" (UID: \"3f63cc50-f78f-4814-b47d-93b0efefafa1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf" Jul 10 18:25:14.382825 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.382592 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drj46\" (UniqueName: \"kubernetes.io/projected/3f63cc50-f78f-4814-b47d-93b0efefafa1-kube-api-access-drj46\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf\" (UID: \"3f63cc50-f78f-4814-b47d-93b0efefafa1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf" Jul 10 18:25:14.382825 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.382815 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f63cc50-f78f-4814-b47d-93b0efefafa1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf\" (UID: \"3f63cc50-f78f-4814-b47d-93b0efefafa1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf" Jul 10 18:25:14.382917 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.382847 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f63cc50-f78f-4814-b47d-93b0efefafa1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf\" (UID: \"3f63cc50-f78f-4814-b47d-93b0efefafa1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf" Jul 10 18:25:14.391747 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.391724 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drj46\" (UniqueName: \"kubernetes.io/projected/3f63cc50-f78f-4814-b47d-93b0efefafa1-kube-api-access-drj46\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf\" (UID: \"3f63cc50-f78f-4814-b47d-93b0efefafa1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf" Jul 10 18:25:14.483466 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.483361 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7064372d-282f-48d8-a557-e4a5111bca35-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6\" (UID: \"7064372d-282f-48d8-a557-e4a5111bca35\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6" Jul 10 18:25:14.483466 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.483422 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lwtl\" (UniqueName: \"kubernetes.io/projected/7064372d-282f-48d8-a557-e4a5111bca35-kube-api-access-5lwtl\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6\" (UID: \"7064372d-282f-48d8-a557-e4a5111bca35\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6" Jul 10 18:25:14.483466 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.483448 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7064372d-282f-48d8-a557-e4a5111bca35-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6\" (UID: \"7064372d-282f-48d8-a557-e4a5111bca35\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6" Jul 10 18:25:14.483807 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.483788 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7064372d-282f-48d8-a557-e4a5111bca35-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6\" (UID: \"7064372d-282f-48d8-a557-e4a5111bca35\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6" Jul 10 18:25:14.483861 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.483844 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7064372d-282f-48d8-a557-e4a5111bca35-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6\" (UID: \"7064372d-282f-48d8-a557-e4a5111bca35\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6" Jul 10 18:25:14.491418 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.491394 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lwtl\" (UniqueName: \"kubernetes.io/projected/7064372d-282f-48d8-a557-e4a5111bca35-kube-api-access-5lwtl\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6\" (UID: \"7064372d-282f-48d8-a557-e4a5111bca35\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6" Jul 10 18:25:14.522021 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.521986 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf" Jul 10 18:25:14.618317 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.618275 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6" Jul 10 18:25:14.654900 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.654867 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf"] Jul 10 18:25:14.657002 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:25:14.656977 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f63cc50_f78f_4814_b47d_93b0efefafa1.slice/crio-d384c6380a60c425d85fdd2b36ef0f200c61fa4335b2378004c2f3838a3ac831 WatchSource:0}: Error finding container d384c6380a60c425d85fdd2b36ef0f200c61fa4335b2378004c2f3838a3ac831: Status 404 returned error can't find the container with id d384c6380a60c425d85fdd2b36ef0f200c61fa4335b2378004c2f3838a3ac831 Jul 10 18:25:14.665039 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.665014 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf" event={"ID":"3f63cc50-f78f-4814-b47d-93b0efefafa1","Type":"ContainerStarted","Data":"d384c6380a60c425d85fdd2b36ef0f200c61fa4335b2378004c2f3838a3ac831"} Jul 10 18:25:14.727975 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.727537 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4"] Jul 10 18:25:14.730489 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.730464 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4" Jul 10 18:25:14.745419 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.745390 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4"] Jul 10 18:25:14.760739 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.760714 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6"] Jul 10 18:25:14.763208 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:25:14.763177 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7064372d_282f_48d8_a557_e4a5111bca35.slice/crio-166c7c471909b7274175bdf3e08c228ca7ba44627fec4962124c2fb331ee0b9d WatchSource:0}: Error finding container 166c7c471909b7274175bdf3e08c228ca7ba44627fec4962124c2fb331ee0b9d: Status 404 returned error can't find the container with id 166c7c471909b7274175bdf3e08c228ca7ba44627fec4962124c2fb331ee0b9d Jul 10 18:25:14.886968 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.886922 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9a4c970-a93f-48fb-82ac-6ecc1663d4b7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4\" (UID: \"a9a4c970-a93f-48fb-82ac-6ecc1663d4b7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4" Jul 10 18:25:14.886968 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.886964 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9a4c970-a93f-48fb-82ac-6ecc1663d4b7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4\" (UID: \"a9a4c970-a93f-48fb-82ac-6ecc1663d4b7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4" Jul 10 18:25:14.887244 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.886990 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgs92\" (UniqueName: \"kubernetes.io/projected/a9a4c970-a93f-48fb-82ac-6ecc1663d4b7-kube-api-access-qgs92\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4\" (UID: \"a9a4c970-a93f-48fb-82ac-6ecc1663d4b7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4" Jul 10 18:25:14.987637 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.987534 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9a4c970-a93f-48fb-82ac-6ecc1663d4b7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4\" (UID: \"a9a4c970-a93f-48fb-82ac-6ecc1663d4b7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4" Jul 10 18:25:14.987637 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.987575 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgs92\" (UniqueName: \"kubernetes.io/projected/a9a4c970-a93f-48fb-82ac-6ecc1663d4b7-kube-api-access-qgs92\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4\" (UID: \"a9a4c970-a93f-48fb-82ac-6ecc1663d4b7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4" Jul 10 18:25:14.987816 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.987661 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9a4c970-a93f-48fb-82ac-6ecc1663d4b7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4\" (UID: \"a9a4c970-a93f-48fb-82ac-6ecc1663d4b7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4" Jul 10 18:25:14.987964 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.987946 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9a4c970-a93f-48fb-82ac-6ecc1663d4b7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4\" (UID: \"a9a4c970-a93f-48fb-82ac-6ecc1663d4b7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4" Jul 10 18:25:14.988008 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.987976 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9a4c970-a93f-48fb-82ac-6ecc1663d4b7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4\" (UID: \"a9a4c970-a93f-48fb-82ac-6ecc1663d4b7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4" Jul 10 18:25:14.996066 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:14.996044 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgs92\" (UniqueName: \"kubernetes.io/projected/a9a4c970-a93f-48fb-82ac-6ecc1663d4b7-kube-api-access-qgs92\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4\" (UID: \"a9a4c970-a93f-48fb-82ac-6ecc1663d4b7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4" Jul 10 18:25:15.041073 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.041043 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4" Jul 10 18:25:15.127811 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.127775 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4"] Jul 10 18:25:15.133018 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.132993 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4" Jul 10 18:25:15.141540 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.141312 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4"] Jul 10 18:25:15.178852 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.178817 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4"] Jul 10 18:25:15.181405 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:25:15.181375 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9a4c970_a93f_48fb_82ac_6ecc1663d4b7.slice/crio-cfe12bb030b8642c0772b93db2dc3e9e344c66ad0f2159806c7413963a9ddea8 WatchSource:0}: Error finding container cfe12bb030b8642c0772b93db2dc3e9e344c66ad0f2159806c7413963a9ddea8: Status 404 returned error can't find the container with id cfe12bb030b8642c0772b93db2dc3e9e344c66ad0f2159806c7413963a9ddea8 Jul 10 18:25:15.290375 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.290339 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70818f90-e5b1-4107-b55a-8e7a53b96814-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4\" (UID: \"70818f90-e5b1-4107-b55a-8e7a53b96814\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4" Jul 10 18:25:15.290375 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.290380 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70818f90-e5b1-4107-b55a-8e7a53b96814-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4\" (UID: \"70818f90-e5b1-4107-b55a-8e7a53b96814\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4" Jul 10 18:25:15.290815 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.290416 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47z4h\" (UniqueName: \"kubernetes.io/projected/70818f90-e5b1-4107-b55a-8e7a53b96814-kube-api-access-47z4h\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4\" (UID: \"70818f90-e5b1-4107-b55a-8e7a53b96814\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4" Jul 10 18:25:15.391362 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.391323 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70818f90-e5b1-4107-b55a-8e7a53b96814-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4\" (UID: \"70818f90-e5b1-4107-b55a-8e7a53b96814\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4" Jul 10 18:25:15.391362 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.391367 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70818f90-e5b1-4107-b55a-8e7a53b96814-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4\" (UID: \"70818f90-e5b1-4107-b55a-8e7a53b96814\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4" Jul 10 18:25:15.391576 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.391401 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47z4h\" (UniqueName: \"kubernetes.io/projected/70818f90-e5b1-4107-b55a-8e7a53b96814-kube-api-access-47z4h\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4\" (UID: \"70818f90-e5b1-4107-b55a-8e7a53b96814\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4" Jul 10 18:25:15.391860 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.391841 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70818f90-e5b1-4107-b55a-8e7a53b96814-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4\" (UID: \"70818f90-e5b1-4107-b55a-8e7a53b96814\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4" Jul 10 18:25:15.391907 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.391885 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70818f90-e5b1-4107-b55a-8e7a53b96814-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4\" (UID: \"70818f90-e5b1-4107-b55a-8e7a53b96814\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4" Jul 10 18:25:15.400348 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.400317 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47z4h\" (UniqueName: \"kubernetes.io/projected/70818f90-e5b1-4107-b55a-8e7a53b96814-kube-api-access-47z4h\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4\" (UID: \"70818f90-e5b1-4107-b55a-8e7a53b96814\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4" Jul 10 18:25:15.445306 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.445269 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4" Jul 10 18:25:15.578677 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.578648 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4"] Jul 10 18:25:15.581149 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:25:15.581123 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70818f90_e5b1_4107_b55a_8e7a53b96814.slice/crio-3b7f376890de05dd8a238ae8f8225aa5b825fc769fba5c351f9b67a9973dfa2b WatchSource:0}: Error finding container 3b7f376890de05dd8a238ae8f8225aa5b825fc769fba5c351f9b67a9973dfa2b: Status 404 returned error can't find the container with id 3b7f376890de05dd8a238ae8f8225aa5b825fc769fba5c351f9b67a9973dfa2b Jul 10 18:25:15.671268 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.671235 2571 generic.go:356] "Generic (PLEG): container finished" podID="3f63cc50-f78f-4814-b47d-93b0efefafa1" containerID="23bb6fa236a38c97b31bc7d2a77a7e4452b11631f6b6406a387151a1ac68483e" exitCode=0 Jul 10 18:25:15.671434 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.671334 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf" event={"ID":"3f63cc50-f78f-4814-b47d-93b0efefafa1","Type":"ContainerDied","Data":"23bb6fa236a38c97b31bc7d2a77a7e4452b11631f6b6406a387151a1ac68483e"} Jul 10 18:25:15.673055 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.673030 2571 generic.go:356] "Generic (PLEG): container finished" podID="a9a4c970-a93f-48fb-82ac-6ecc1663d4b7" containerID="bbe9ee271a2167426857efbc0e8469b2d38b9564b1373bb3c652a75f685fa76a" exitCode=0 Jul 10 18:25:15.673147 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.673115 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4" event={"ID":"a9a4c970-a93f-48fb-82ac-6ecc1663d4b7","Type":"ContainerDied","Data":"bbe9ee271a2167426857efbc0e8469b2d38b9564b1373bb3c652a75f685fa76a"} Jul 10 18:25:15.673198 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.673169 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4" event={"ID":"a9a4c970-a93f-48fb-82ac-6ecc1663d4b7","Type":"ContainerStarted","Data":"cfe12bb030b8642c0772b93db2dc3e9e344c66ad0f2159806c7413963a9ddea8"} Jul 10 18:25:15.674981 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.674837 2571 generic.go:356] "Generic (PLEG): container finished" podID="7064372d-282f-48d8-a557-e4a5111bca35" containerID="8b17f8754b5cdc9ecb1706b4c2d1a0f1f369bbc85b1da1f810898251695eb29c" exitCode=0 Jul 10 18:25:15.674981 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.674925 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6" event={"ID":"7064372d-282f-48d8-a557-e4a5111bca35","Type":"ContainerDied","Data":"8b17f8754b5cdc9ecb1706b4c2d1a0f1f369bbc85b1da1f810898251695eb29c"} Jul 10 18:25:15.675090 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.674980 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6" event={"ID":"7064372d-282f-48d8-a557-e4a5111bca35","Type":"ContainerStarted","Data":"166c7c471909b7274175bdf3e08c228ca7ba44627fec4962124c2fb331ee0b9d"} Jul 10 18:25:15.676834 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.676688 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4" event={"ID":"70818f90-e5b1-4107-b55a-8e7a53b96814","Type":"ContainerStarted","Data":"17c913dd4fc1cc66fb01990230a450a27fe56d062ac23d33ecf702f9be1cd469"} Jul 10 18:25:15.676834 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:15.676736 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4" event={"ID":"70818f90-e5b1-4107-b55a-8e7a53b96814","Type":"ContainerStarted","Data":"3b7f376890de05dd8a238ae8f8225aa5b825fc769fba5c351f9b67a9973dfa2b"} Jul 10 18:25:16.683082 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:16.683054 2571 generic.go:356] "Generic (PLEG): container finished" podID="a9a4c970-a93f-48fb-82ac-6ecc1663d4b7" containerID="e65646b8838f91d66c795c7e501e8382873d812fe5a8ad464082e67c815e3fe9" exitCode=0 Jul 10 18:25:16.683434 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:16.683133 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4" event={"ID":"a9a4c970-a93f-48fb-82ac-6ecc1663d4b7","Type":"ContainerDied","Data":"e65646b8838f91d66c795c7e501e8382873d812fe5a8ad464082e67c815e3fe9"} Jul 10 18:25:16.684769 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:16.684751 2571 generic.go:356] "Generic (PLEG): container finished" podID="70818f90-e5b1-4107-b55a-8e7a53b96814" containerID="17c913dd4fc1cc66fb01990230a450a27fe56d062ac23d33ecf702f9be1cd469" exitCode=0 Jul 10 18:25:16.684874 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:16.684826 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4" event={"ID":"70818f90-e5b1-4107-b55a-8e7a53b96814","Type":"ContainerDied","Data":"17c913dd4fc1cc66fb01990230a450a27fe56d062ac23d33ecf702f9be1cd469"} Jul 10 18:25:17.691066 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:17.690972 2571 generic.go:356] "Generic (PLEG): container finished" podID="3f63cc50-f78f-4814-b47d-93b0efefafa1" containerID="606caabf6b8f5655c0675a0120f427feee01c5f59068ff878a0762ab964df464" exitCode=0 Jul 10 18:25:17.691504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:17.691063 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf" event={"ID":"3f63cc50-f78f-4814-b47d-93b0efefafa1","Type":"ContainerDied","Data":"606caabf6b8f5655c0675a0120f427feee01c5f59068ff878a0762ab964df464"} Jul 10 18:25:17.699203 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:17.699180 2571 generic.go:356] "Generic (PLEG): container finished" podID="a9a4c970-a93f-48fb-82ac-6ecc1663d4b7" containerID="aa65434126bf98f0c0ccd286b582b51b78d2c1b3890120dea548b1136b4acb89" exitCode=0 Jul 10 18:25:17.699289 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:17.699257 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4" event={"ID":"a9a4c970-a93f-48fb-82ac-6ecc1663d4b7","Type":"ContainerDied","Data":"aa65434126bf98f0c0ccd286b582b51b78d2c1b3890120dea548b1136b4acb89"} Jul 10 18:25:17.701256 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:17.701230 2571 generic.go:356] "Generic (PLEG): container finished" podID="7064372d-282f-48d8-a557-e4a5111bca35" containerID="55515fbad2c8afaae1e244ef7f51d2acca67cc9748be2e7f8d59bb411ae15ba3" exitCode=0 Jul 10 18:25:17.701361 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:17.701292 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6" event={"ID":"7064372d-282f-48d8-a557-e4a5111bca35","Type":"ContainerDied","Data":"55515fbad2c8afaae1e244ef7f51d2acca67cc9748be2e7f8d59bb411ae15ba3"} Jul 10 18:25:18.708173 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:18.708079 2571 generic.go:356] "Generic (PLEG): container finished" podID="3f63cc50-f78f-4814-b47d-93b0efefafa1" containerID="d0696ac685cf3bce101bea85005ab74c1a169c46d58df1cf6ed5dab933edd35b" exitCode=0 Jul 10 18:25:18.708633 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:18.708162 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf" event={"ID":"3f63cc50-f78f-4814-b47d-93b0efefafa1","Type":"ContainerDied","Data":"d0696ac685cf3bce101bea85005ab74c1a169c46d58df1cf6ed5dab933edd35b"} Jul 10 18:25:18.710246 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:18.710218 2571 generic.go:356] "Generic (PLEG): container finished" podID="7064372d-282f-48d8-a557-e4a5111bca35" containerID="f9620fca80ee619bebe88e486d0ac309d2bf3ffd54d986627fc06622a0524710" exitCode=0 Jul 10 18:25:18.710378 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:18.710305 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6" event={"ID":"7064372d-282f-48d8-a557-e4a5111bca35","Type":"ContainerDied","Data":"f9620fca80ee619bebe88e486d0ac309d2bf3ffd54d986627fc06622a0524710"} Jul 10 18:25:18.712119 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:18.712094 2571 generic.go:356] "Generic (PLEG): container finished" podID="70818f90-e5b1-4107-b55a-8e7a53b96814" containerID="991445e3af36d2cb7aeb78c4960d466487104298a60167458caaa541124f023f" exitCode=0 Jul 10 18:25:18.712212 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:18.712170 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4" event={"ID":"70818f90-e5b1-4107-b55a-8e7a53b96814","Type":"ContainerDied","Data":"991445e3af36d2cb7aeb78c4960d466487104298a60167458caaa541124f023f"} Jul 10 18:25:18.840767 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:18.840738 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4" Jul 10 18:25:18.922397 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:18.922363 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9a4c970-a93f-48fb-82ac-6ecc1663d4b7-bundle\") pod \"a9a4c970-a93f-48fb-82ac-6ecc1663d4b7\" (UID: \"a9a4c970-a93f-48fb-82ac-6ecc1663d4b7\") " Jul 10 18:25:18.922556 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:18.922424 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgs92\" (UniqueName: \"kubernetes.io/projected/a9a4c970-a93f-48fb-82ac-6ecc1663d4b7-kube-api-access-qgs92\") pod \"a9a4c970-a93f-48fb-82ac-6ecc1663d4b7\" (UID: \"a9a4c970-a93f-48fb-82ac-6ecc1663d4b7\") " Jul 10 18:25:18.922556 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:18.922491 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9a4c970-a93f-48fb-82ac-6ecc1663d4b7-util\") pod \"a9a4c970-a93f-48fb-82ac-6ecc1663d4b7\" (UID: \"a9a4c970-a93f-48fb-82ac-6ecc1663d4b7\") " Jul 10 18:25:18.922888 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:18.922860 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9a4c970-a93f-48fb-82ac-6ecc1663d4b7-bundle" (OuterVolumeSpecName: "bundle") pod "a9a4c970-a93f-48fb-82ac-6ecc1663d4b7" (UID: "a9a4c970-a93f-48fb-82ac-6ecc1663d4b7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:25:18.924485 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:18.924467 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9a4c970-a93f-48fb-82ac-6ecc1663d4b7-kube-api-access-qgs92" (OuterVolumeSpecName: "kube-api-access-qgs92") pod "a9a4c970-a93f-48fb-82ac-6ecc1663d4b7" (UID: "a9a4c970-a93f-48fb-82ac-6ecc1663d4b7"). InnerVolumeSpecName "kube-api-access-qgs92". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:25:18.927813 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:18.927789 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9a4c970-a93f-48fb-82ac-6ecc1663d4b7-util" (OuterVolumeSpecName: "util") pod "a9a4c970-a93f-48fb-82ac-6ecc1663d4b7" (UID: "a9a4c970-a93f-48fb-82ac-6ecc1663d4b7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:25:19.023211 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:19.023180 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9a4c970-a93f-48fb-82ac-6ecc1663d4b7-util\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:25:19.023211 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:19.023209 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9a4c970-a93f-48fb-82ac-6ecc1663d4b7-bundle\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:25:19.023368 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:19.023219 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgs92\" (UniqueName: \"kubernetes.io/projected/a9a4c970-a93f-48fb-82ac-6ecc1663d4b7-kube-api-access-qgs92\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:25:19.718608 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:19.718582 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4" Jul 10 18:25:19.718608 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:19.718592 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4" event={"ID":"a9a4c970-a93f-48fb-82ac-6ecc1663d4b7","Type":"ContainerDied","Data":"cfe12bb030b8642c0772b93db2dc3e9e344c66ad0f2159806c7413963a9ddea8"} Jul 10 18:25:19.719073 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:19.718656 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cfe12bb030b8642c0772b93db2dc3e9e344c66ad0f2159806c7413963a9ddea8" Jul 10 18:25:19.721105 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:19.721077 2571 generic.go:356] "Generic (PLEG): container finished" podID="70818f90-e5b1-4107-b55a-8e7a53b96814" containerID="ae7a806db8893042fa84e06333adad71757ed791c39a3c3f0a4c4feeea2c7572" exitCode=0 Jul 10 18:25:19.721233 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:19.721155 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4" event={"ID":"70818f90-e5b1-4107-b55a-8e7a53b96814","Type":"ContainerDied","Data":"ae7a806db8893042fa84e06333adad71757ed791c39a3c3f0a4c4feeea2c7572"} Jul 10 18:25:19.851747 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:19.851720 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6" Jul 10 18:25:19.882990 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:19.882962 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf" Jul 10 18:25:20.031918 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.031884 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7064372d-282f-48d8-a557-e4a5111bca35-util\") pod \"7064372d-282f-48d8-a557-e4a5111bca35\" (UID: \"7064372d-282f-48d8-a557-e4a5111bca35\") " Jul 10 18:25:20.032080 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.031930 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7064372d-282f-48d8-a557-e4a5111bca35-bundle\") pod \"7064372d-282f-48d8-a557-e4a5111bca35\" (UID: \"7064372d-282f-48d8-a557-e4a5111bca35\") " Jul 10 18:25:20.032080 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.031949 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lwtl\" (UniqueName: \"kubernetes.io/projected/7064372d-282f-48d8-a557-e4a5111bca35-kube-api-access-5lwtl\") pod \"7064372d-282f-48d8-a557-e4a5111bca35\" (UID: \"7064372d-282f-48d8-a557-e4a5111bca35\") " Jul 10 18:25:20.032080 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.032002 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drj46\" (UniqueName: \"kubernetes.io/projected/3f63cc50-f78f-4814-b47d-93b0efefafa1-kube-api-access-drj46\") pod \"3f63cc50-f78f-4814-b47d-93b0efefafa1\" (UID: \"3f63cc50-f78f-4814-b47d-93b0efefafa1\") " Jul 10 18:25:20.032080 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.032020 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f63cc50-f78f-4814-b47d-93b0efefafa1-bundle\") pod \"3f63cc50-f78f-4814-b47d-93b0efefafa1\" (UID: \"3f63cc50-f78f-4814-b47d-93b0efefafa1\") " Jul 10 18:25:20.032388 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.032084 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f63cc50-f78f-4814-b47d-93b0efefafa1-util\") pod \"3f63cc50-f78f-4814-b47d-93b0efefafa1\" (UID: \"3f63cc50-f78f-4814-b47d-93b0efefafa1\") " Jul 10 18:25:20.032573 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.032545 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7064372d-282f-48d8-a557-e4a5111bca35-bundle" (OuterVolumeSpecName: "bundle") pod "7064372d-282f-48d8-a557-e4a5111bca35" (UID: "7064372d-282f-48d8-a557-e4a5111bca35"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:25:20.032740 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.032710 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f63cc50-f78f-4814-b47d-93b0efefafa1-bundle" (OuterVolumeSpecName: "bundle") pod "3f63cc50-f78f-4814-b47d-93b0efefafa1" (UID: "3f63cc50-f78f-4814-b47d-93b0efefafa1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:25:20.034137 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.034117 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f63cc50-f78f-4814-b47d-93b0efefafa1-kube-api-access-drj46" (OuterVolumeSpecName: "kube-api-access-drj46") pod "3f63cc50-f78f-4814-b47d-93b0efefafa1" (UID: "3f63cc50-f78f-4814-b47d-93b0efefafa1"). InnerVolumeSpecName "kube-api-access-drj46". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:25:20.034196 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.034137 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7064372d-282f-48d8-a557-e4a5111bca35-kube-api-access-5lwtl" (OuterVolumeSpecName: "kube-api-access-5lwtl") pod "7064372d-282f-48d8-a557-e4a5111bca35" (UID: "7064372d-282f-48d8-a557-e4a5111bca35"). InnerVolumeSpecName "kube-api-access-5lwtl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:25:20.133017 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.132980 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drj46\" (UniqueName: \"kubernetes.io/projected/3f63cc50-f78f-4814-b47d-93b0efefafa1-kube-api-access-drj46\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:25:20.133017 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.133011 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f63cc50-f78f-4814-b47d-93b0efefafa1-bundle\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:25:20.133017 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.133021 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7064372d-282f-48d8-a557-e4a5111bca35-bundle\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:25:20.133017 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.133029 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lwtl\" (UniqueName: \"kubernetes.io/projected/7064372d-282f-48d8-a557-e4a5111bca35-kube-api-access-5lwtl\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:25:20.186354 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.186305 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7064372d-282f-48d8-a557-e4a5111bca35-util" (OuterVolumeSpecName: "util") pod "7064372d-282f-48d8-a557-e4a5111bca35" (UID: "7064372d-282f-48d8-a557-e4a5111bca35"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:25:20.191682 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.191604 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f63cc50-f78f-4814-b47d-93b0efefafa1-util" (OuterVolumeSpecName: "util") pod "3f63cc50-f78f-4814-b47d-93b0efefafa1" (UID: "3f63cc50-f78f-4814-b47d-93b0efefafa1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:25:20.233716 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.233664 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f63cc50-f78f-4814-b47d-93b0efefafa1-util\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:25:20.233716 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.233710 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7064372d-282f-48d8-a557-e4a5111bca35-util\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:25:20.728217 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.728112 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf" Jul 10 18:25:20.728217 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.728129 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf" event={"ID":"3f63cc50-f78f-4814-b47d-93b0efefafa1","Type":"ContainerDied","Data":"d384c6380a60c425d85fdd2b36ef0f200c61fa4335b2378004c2f3838a3ac831"} Jul 10 18:25:20.728217 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.728188 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d384c6380a60c425d85fdd2b36ef0f200c61fa4335b2378004c2f3838a3ac831" Jul 10 18:25:20.730065 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.730046 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6" Jul 10 18:25:20.730192 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.730085 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6" event={"ID":"7064372d-282f-48d8-a557-e4a5111bca35","Type":"ContainerDied","Data":"166c7c471909b7274175bdf3e08c228ca7ba44627fec4962124c2fb331ee0b9d"} Jul 10 18:25:20.730192 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.730125 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="166c7c471909b7274175bdf3e08c228ca7ba44627fec4962124c2fb331ee0b9d" Jul 10 18:25:20.866956 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:20.866932 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4" Jul 10 18:25:21.040222 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:21.040190 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70818f90-e5b1-4107-b55a-8e7a53b96814-util\") pod \"70818f90-e5b1-4107-b55a-8e7a53b96814\" (UID: \"70818f90-e5b1-4107-b55a-8e7a53b96814\") " Jul 10 18:25:21.040385 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:21.040293 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47z4h\" (UniqueName: \"kubernetes.io/projected/70818f90-e5b1-4107-b55a-8e7a53b96814-kube-api-access-47z4h\") pod \"70818f90-e5b1-4107-b55a-8e7a53b96814\" (UID: \"70818f90-e5b1-4107-b55a-8e7a53b96814\") " Jul 10 18:25:21.040385 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:21.040329 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70818f90-e5b1-4107-b55a-8e7a53b96814-bundle\") pod \"70818f90-e5b1-4107-b55a-8e7a53b96814\" (UID: \"70818f90-e5b1-4107-b55a-8e7a53b96814\") " Jul 10 18:25:21.040896 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:21.040868 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70818f90-e5b1-4107-b55a-8e7a53b96814-bundle" (OuterVolumeSpecName: "bundle") pod "70818f90-e5b1-4107-b55a-8e7a53b96814" (UID: "70818f90-e5b1-4107-b55a-8e7a53b96814"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:25:21.042337 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:21.042314 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70818f90-e5b1-4107-b55a-8e7a53b96814-kube-api-access-47z4h" (OuterVolumeSpecName: "kube-api-access-47z4h") pod "70818f90-e5b1-4107-b55a-8e7a53b96814" (UID: "70818f90-e5b1-4107-b55a-8e7a53b96814"). InnerVolumeSpecName "kube-api-access-47z4h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:25:21.140920 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:21.140879 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47z4h\" (UniqueName: \"kubernetes.io/projected/70818f90-e5b1-4107-b55a-8e7a53b96814-kube-api-access-47z4h\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:25:21.140920 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:21.140912 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70818f90-e5b1-4107-b55a-8e7a53b96814-bundle\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:25:21.142553 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:21.142524 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70818f90-e5b1-4107-b55a-8e7a53b96814-util" (OuterVolumeSpecName: "util") pod "70818f90-e5b1-4107-b55a-8e7a53b96814" (UID: "70818f90-e5b1-4107-b55a-8e7a53b96814"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:25:21.241710 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:21.241672 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70818f90-e5b1-4107-b55a-8e7a53b96814-util\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:25:21.737085 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:21.737059 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4" Jul 10 18:25:21.737541 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:21.737061 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4" event={"ID":"70818f90-e5b1-4107-b55a-8e7a53b96814","Type":"ContainerDied","Data":"3b7f376890de05dd8a238ae8f8225aa5b825fc769fba5c351f9b67a9973dfa2b"} Jul 10 18:25:21.737541 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:21.737177 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b7f376890de05dd8a238ae8f8225aa5b825fc769fba5c351f9b67a9973dfa2b" Jul 10 18:25:29.704998 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.704964 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw6ph"] Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705260 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f63cc50-f78f-4814-b47d-93b0efefafa1" containerName="util" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705270 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f63cc50-f78f-4814-b47d-93b0efefafa1" containerName="util" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705277 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7064372d-282f-48d8-a557-e4a5111bca35" containerName="pull" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705283 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7064372d-282f-48d8-a557-e4a5111bca35" containerName="pull" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705289 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70818f90-e5b1-4107-b55a-8e7a53b96814" containerName="pull" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705294 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="70818f90-e5b1-4107-b55a-8e7a53b96814" containerName="pull" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705302 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f63cc50-f78f-4814-b47d-93b0efefafa1" containerName="pull" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705307 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f63cc50-f78f-4814-b47d-93b0efefafa1" containerName="pull" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705315 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70818f90-e5b1-4107-b55a-8e7a53b96814" containerName="extract" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705320 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="70818f90-e5b1-4107-b55a-8e7a53b96814" containerName="extract" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705328 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f63cc50-f78f-4814-b47d-93b0efefafa1" containerName="extract" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705333 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f63cc50-f78f-4814-b47d-93b0efefafa1" containerName="extract" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705339 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7064372d-282f-48d8-a557-e4a5111bca35" containerName="extract" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705344 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7064372d-282f-48d8-a557-e4a5111bca35" containerName="extract" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705351 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70818f90-e5b1-4107-b55a-8e7a53b96814" containerName="util" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705355 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="70818f90-e5b1-4107-b55a-8e7a53b96814" containerName="util" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705360 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9a4c970-a93f-48fb-82ac-6ecc1663d4b7" containerName="pull" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705364 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9a4c970-a93f-48fb-82ac-6ecc1663d4b7" containerName="pull" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705370 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9a4c970-a93f-48fb-82ac-6ecc1663d4b7" containerName="util" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705375 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9a4c970-a93f-48fb-82ac-6ecc1663d4b7" containerName="util" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705383 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7064372d-282f-48d8-a557-e4a5111bca35" containerName="util" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705388 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7064372d-282f-48d8-a557-e4a5111bca35" containerName="util" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705392 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9a4c970-a93f-48fb-82ac-6ecc1663d4b7" containerName="extract" Jul 10 18:25:29.705399 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705396 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9a4c970-a93f-48fb-82ac-6ecc1663d4b7" containerName="extract" Jul 10 18:25:29.706151 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705451 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9a4c970-a93f-48fb-82ac-6ecc1663d4b7" containerName="extract" Jul 10 18:25:29.706151 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705459 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f63cc50-f78f-4814-b47d-93b0efefafa1" containerName="extract" Jul 10 18:25:29.706151 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705466 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7064372d-282f-48d8-a557-e4a5111bca35" containerName="extract" Jul 10 18:25:29.706151 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.705472 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="70818f90-e5b1-4107-b55a-8e7a53b96814" containerName="extract" Jul 10 18:25:29.711080 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.711061 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw6ph" Jul 10 18:25:29.717905 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.717883 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 10 18:25:29.718309 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.718287 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-87pnr\"" Jul 10 18:25:29.722864 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.722844 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw6ph"] Jul 10 18:25:29.810555 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.810510 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-662nl\" (UniqueName: \"kubernetes.io/projected/3fac146c-8090-4f9c-bcc4-ca041ffbe7d3-kube-api-access-662nl\") pod \"dns-operator-controller-manager-864dd78c4c-qw6ph\" (UID: \"3fac146c-8090-4f9c-bcc4-ca041ffbe7d3\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw6ph" Jul 10 18:25:29.910938 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.910885 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-662nl\" (UniqueName: \"kubernetes.io/projected/3fac146c-8090-4f9c-bcc4-ca041ffbe7d3-kube-api-access-662nl\") pod \"dns-operator-controller-manager-864dd78c4c-qw6ph\" (UID: \"3fac146c-8090-4f9c-bcc4-ca041ffbe7d3\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw6ph" Jul 10 18:25:29.921098 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:29.921060 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-662nl\" (UniqueName: \"kubernetes.io/projected/3fac146c-8090-4f9c-bcc4-ca041ffbe7d3-kube-api-access-662nl\") pod \"dns-operator-controller-manager-864dd78c4c-qw6ph\" (UID: \"3fac146c-8090-4f9c-bcc4-ca041ffbe7d3\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw6ph" Jul 10 18:25:30.021070 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:30.021021 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw6ph" Jul 10 18:25:30.157236 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:30.157207 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw6ph"] Jul 10 18:25:30.160247 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:25:30.160220 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fac146c_8090_4f9c_bcc4_ca041ffbe7d3.slice/crio-041b5efa4ea5f1170f6b11faaf8e5f564051fb1fc9aeb3263583ffec1311bde5 WatchSource:0}: Error finding container 041b5efa4ea5f1170f6b11faaf8e5f564051fb1fc9aeb3263583ffec1311bde5: Status 404 returned error can't find the container with id 041b5efa4ea5f1170f6b11faaf8e5f564051fb1fc9aeb3263583ffec1311bde5 Jul 10 18:25:30.776278 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:30.776247 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw6ph" event={"ID":"3fac146c-8090-4f9c-bcc4-ca041ffbe7d3","Type":"ContainerStarted","Data":"041b5efa4ea5f1170f6b11faaf8e5f564051fb1fc9aeb3263583ffec1311bde5"} Jul 10 18:25:32.787836 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:32.787796 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw6ph" event={"ID":"3fac146c-8090-4f9c-bcc4-ca041ffbe7d3","Type":"ContainerStarted","Data":"c762d23efe4450931b7f27786d5aac706497edf4baa4ebf66154a2f9f673ded5"} Jul 10 18:25:32.788229 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:32.787946 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw6ph" Jul 10 18:25:32.807647 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:32.807576 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw6ph" podStartSLOduration=1.727358819 podStartE2EDuration="3.807558008s" totalImagesPullingTime="2.080199189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:25:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:25:32.8052027 +0000 UTC m=+489.704471827" watchObservedRunningTime="2026-07-10 18:25:32.807558008 +0000 UTC m=+489.706827115" Jul 10 18:25:33.789443 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:33.789412 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lkjnp"] Jul 10 18:25:33.793125 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:33.793104 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lkjnp" Jul 10 18:25:33.796101 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:33.796083 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-4w4qv\"" Jul 10 18:25:33.802226 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:33.802203 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lkjnp"] Jul 10 18:25:33.943573 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:33.943540 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvmj9\" (UniqueName: \"kubernetes.io/projected/f2102ee6-215c-4455-9c64-48d52a6842da-kube-api-access-pvmj9\") pod \"authorino-operator-75cc8d5cb7-lkjnp\" (UID: \"f2102ee6-215c-4455-9c64-48d52a6842da\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lkjnp" Jul 10 18:25:34.044233 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:34.044156 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvmj9\" (UniqueName: \"kubernetes.io/projected/f2102ee6-215c-4455-9c64-48d52a6842da-kube-api-access-pvmj9\") pod \"authorino-operator-75cc8d5cb7-lkjnp\" (UID: \"f2102ee6-215c-4455-9c64-48d52a6842da\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lkjnp" Jul 10 18:25:34.066669 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:34.066641 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvmj9\" (UniqueName: \"kubernetes.io/projected/f2102ee6-215c-4455-9c64-48d52a6842da-kube-api-access-pvmj9\") pod \"authorino-operator-75cc8d5cb7-lkjnp\" (UID: \"f2102ee6-215c-4455-9c64-48d52a6842da\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lkjnp" Jul 10 18:25:34.103482 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:34.103437 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lkjnp" Jul 10 18:25:34.240859 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:34.240835 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lkjnp"] Jul 10 18:25:34.243033 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:25:34.243001 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2102ee6_215c_4455_9c64_48d52a6842da.slice/crio-1908f9cf5c2ba828076cdd157282fc1e0148a59912b75b1690ba7cb67023c5eb WatchSource:0}: Error finding container 1908f9cf5c2ba828076cdd157282fc1e0148a59912b75b1690ba7cb67023c5eb: Status 404 returned error can't find the container with id 1908f9cf5c2ba828076cdd157282fc1e0148a59912b75b1690ba7cb67023c5eb Jul 10 18:25:34.798393 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:34.798355 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lkjnp" event={"ID":"f2102ee6-215c-4455-9c64-48d52a6842da","Type":"ContainerStarted","Data":"1908f9cf5c2ba828076cdd157282fc1e0148a59912b75b1690ba7cb67023c5eb"} Jul 10 18:25:36.811211 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:36.811181 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lkjnp" event={"ID":"f2102ee6-215c-4455-9c64-48d52a6842da","Type":"ContainerStarted","Data":"2a4d332403c25515945fb9c710478b37e0e7a8c85e69bb20148083b2fd3ae3b6"} Jul 10 18:25:36.811564 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:36.811300 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lkjnp" Jul 10 18:25:36.829608 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:36.829553 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lkjnp" podStartSLOduration=1.329731447 podStartE2EDuration="3.829537607s" totalImagesPullingTime="2.49980616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:25:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:25:36.827864647 +0000 UTC m=+493.727133788" watchObservedRunningTime="2026-07-10 18:25:36.829537607 +0000 UTC m=+493.728806714" Jul 10 18:25:37.103646 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:37.103548 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244"] Jul 10 18:25:37.106875 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:37.106858 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244" Jul 10 18:25:37.109547 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:37.109530 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-h2tl8\"" Jul 10 18:25:37.124980 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:37.124953 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244"] Jul 10 18:25:37.167244 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:37.167212 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r8pp\" (UniqueName: \"kubernetes.io/projected/3b79561c-7c94-400f-a1cd-7e4b3354facd-kube-api-access-4r8pp\") pod \"limitador-operator-controller-manager-859d7b78bd-bp244\" (UID: \"3b79561c-7c94-400f-a1cd-7e4b3354facd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244" Jul 10 18:25:37.268425 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:37.268384 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r8pp\" (UniqueName: \"kubernetes.io/projected/3b79561c-7c94-400f-a1cd-7e4b3354facd-kube-api-access-4r8pp\") pod \"limitador-operator-controller-manager-859d7b78bd-bp244\" (UID: \"3b79561c-7c94-400f-a1cd-7e4b3354facd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244" Jul 10 18:25:37.277184 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:37.277152 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r8pp\" (UniqueName: \"kubernetes.io/projected/3b79561c-7c94-400f-a1cd-7e4b3354facd-kube-api-access-4r8pp\") pod \"limitador-operator-controller-manager-859d7b78bd-bp244\" (UID: \"3b79561c-7c94-400f-a1cd-7e4b3354facd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244" Jul 10 18:25:37.417421 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:37.417323 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244" Jul 10 18:25:37.557803 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:37.557637 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244"] Jul 10 18:25:37.560136 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:25:37.560105 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b79561c_7c94_400f_a1cd_7e4b3354facd.slice/crio-67c0500c3edfada805ee40621d35af6552ac5e2a80602b5b40feb5f9e398f073 WatchSource:0}: Error finding container 67c0500c3edfada805ee40621d35af6552ac5e2a80602b5b40feb5f9e398f073: Status 404 returned error can't find the container with id 67c0500c3edfada805ee40621d35af6552ac5e2a80602b5b40feb5f9e398f073 Jul 10 18:25:37.817333 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:37.817294 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244" event={"ID":"3b79561c-7c94-400f-a1cd-7e4b3354facd","Type":"ContainerStarted","Data":"67c0500c3edfada805ee40621d35af6552ac5e2a80602b5b40feb5f9e398f073"} Jul 10 18:25:39.829204 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:39.829166 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244" event={"ID":"3b79561c-7c94-400f-a1cd-7e4b3354facd","Type":"ContainerStarted","Data":"539ba0ad068a15339b539fb2ec17eb9feeccbf689f584202be8365135b39025f"} Jul 10 18:25:39.829717 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:39.829236 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244" Jul 10 18:25:39.854243 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:39.854186 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244" podStartSLOduration=0.792442423 podStartE2EDuration="2.854167786s" totalImagesPullingTime="2.061725363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:25:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:25:39.852387956 +0000 UTC m=+496.751657088" watchObservedRunningTime="2026-07-10 18:25:39.854167786 +0000 UTC m=+496.753436893" Jul 10 18:25:41.780276 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:41.780241 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n"] Jul 10 18:25:41.785786 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:41.784304 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:25:41.788320 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:41.788296 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-47r9g\"" Jul 10 18:25:41.798493 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:41.798467 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grksr\" (UniqueName: \"kubernetes.io/projected/4b099513-fbcd-464e-ab3f-695f58af8072-kube-api-access-grksr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9872n\" (UID: \"4b099513-fbcd-464e-ab3f-695f58af8072\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:25:41.798607 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:41.798512 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b099513-fbcd-464e-ab3f-695f58af8072-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9872n\" (UID: \"4b099513-fbcd-464e-ab3f-695f58af8072\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:25:41.808496 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:41.808467 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n"] Jul 10 18:25:41.899739 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:41.899704 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grksr\" (UniqueName: \"kubernetes.io/projected/4b099513-fbcd-464e-ab3f-695f58af8072-kube-api-access-grksr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9872n\" (UID: \"4b099513-fbcd-464e-ab3f-695f58af8072\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:25:41.899918 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:41.899751 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b099513-fbcd-464e-ab3f-695f58af8072-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9872n\" (UID: \"4b099513-fbcd-464e-ab3f-695f58af8072\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:25:41.900109 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:41.900092 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b099513-fbcd-464e-ab3f-695f58af8072-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9872n\" (UID: \"4b099513-fbcd-464e-ab3f-695f58af8072\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:25:41.908889 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:41.908855 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grksr\" (UniqueName: \"kubernetes.io/projected/4b099513-fbcd-464e-ab3f-695f58af8072-kube-api-access-grksr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9872n\" (UID: \"4b099513-fbcd-464e-ab3f-695f58af8072\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:25:42.095601 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:42.095507 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:25:42.232555 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:42.232518 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n"] Jul 10 18:25:42.235803 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:25:42.235775 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b099513_fbcd_464e_ab3f_695f58af8072.slice/crio-b6d99b8b01262490abc95967e9164baeccb0e0e01f274791ab7b94361e7d2123 WatchSource:0}: Error finding container b6d99b8b01262490abc95967e9164baeccb0e0e01f274791ab7b94361e7d2123: Status 404 returned error can't find the container with id b6d99b8b01262490abc95967e9164baeccb0e0e01f274791ab7b94361e7d2123 Jul 10 18:25:42.844340 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:42.844300 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" event={"ID":"4b099513-fbcd-464e-ab3f-695f58af8072","Type":"ContainerStarted","Data":"b6d99b8b01262490abc95967e9164baeccb0e0e01f274791ab7b94361e7d2123"} Jul 10 18:25:43.795122 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:43.795084 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw6ph" Jul 10 18:25:47.820190 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:47.820155 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lkjnp" Jul 10 18:25:49.882220 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:49.882182 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" event={"ID":"4b099513-fbcd-464e-ab3f-695f58af8072","Type":"ContainerStarted","Data":"60e666a83012eae88f2deea03389b1988da418c7d5a5bb645e09e236d5d8e594"} Jul 10 18:25:49.882631 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:49.882330 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:25:49.908141 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:49.908085 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" podStartSLOduration=1.915389357 podStartE2EDuration="8.908067149s" totalImagesPullingTime="6.992677792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:25:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:25:49.907065532 +0000 UTC m=+506.806334639" watchObservedRunningTime="2026-07-10 18:25:49.908067149 +0000 UTC m=+506.807336256" Jul 10 18:25:50.836765 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:50.836733 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244" Jul 10 18:25:52.039450 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:52.039415 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2l5kn"] Jul 10 18:25:52.043156 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:52.043127 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l5kn" Jul 10 18:25:52.045979 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:52.045955 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 10 18:25:52.046130 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:52.046046 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-c6frz\"" Jul 10 18:25:52.046130 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:52.046092 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 10 18:25:52.051875 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:52.051849 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2l5kn"] Jul 10 18:25:52.089273 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:52.089235 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a170d9d2-c194-4b40-af59-80030fdf258f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2l5kn\" (UID: \"a170d9d2-c194-4b40-af59-80030fdf258f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l5kn" Jul 10 18:25:52.089273 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:52.089272 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a170d9d2-c194-4b40-af59-80030fdf258f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2l5kn\" (UID: \"a170d9d2-c194-4b40-af59-80030fdf258f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l5kn" Jul 10 18:25:52.089493 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:52.089292 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng8sz\" (UniqueName: \"kubernetes.io/projected/a170d9d2-c194-4b40-af59-80030fdf258f-kube-api-access-ng8sz\") pod \"kuadrant-console-plugin-66b5774985-2l5kn\" (UID: \"a170d9d2-c194-4b40-af59-80030fdf258f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l5kn" Jul 10 18:25:52.190023 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:52.189985 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a170d9d2-c194-4b40-af59-80030fdf258f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2l5kn\" (UID: \"a170d9d2-c194-4b40-af59-80030fdf258f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l5kn" Jul 10 18:25:52.190023 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:52.190026 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a170d9d2-c194-4b40-af59-80030fdf258f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2l5kn\" (UID: \"a170d9d2-c194-4b40-af59-80030fdf258f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l5kn" Jul 10 18:25:52.190290 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:52.190046 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng8sz\" (UniqueName: \"kubernetes.io/projected/a170d9d2-c194-4b40-af59-80030fdf258f-kube-api-access-ng8sz\") pod \"kuadrant-console-plugin-66b5774985-2l5kn\" (UID: \"a170d9d2-c194-4b40-af59-80030fdf258f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l5kn" Jul 10 18:25:52.190290 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:25:52.190154 2571 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 10 18:25:52.190290 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:25:52.190241 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a170d9d2-c194-4b40-af59-80030fdf258f-plugin-serving-cert podName:a170d9d2-c194-4b40-af59-80030fdf258f nodeName:}" failed. No retries permitted until 2026-07-10 18:25:52.69022035 +0000 UTC m=+509.589489442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/a170d9d2-c194-4b40-af59-80030fdf258f-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-2l5kn" (UID: "a170d9d2-c194-4b40-af59-80030fdf258f") : secret "plugin-serving-cert" not found Jul 10 18:25:52.191072 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:52.191045 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a170d9d2-c194-4b40-af59-80030fdf258f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2l5kn\" (UID: \"a170d9d2-c194-4b40-af59-80030fdf258f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l5kn" Jul 10 18:25:52.199151 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:52.199126 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng8sz\" (UniqueName: \"kubernetes.io/projected/a170d9d2-c194-4b40-af59-80030fdf258f-kube-api-access-ng8sz\") pod \"kuadrant-console-plugin-66b5774985-2l5kn\" (UID: \"a170d9d2-c194-4b40-af59-80030fdf258f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l5kn" Jul 10 18:25:52.700006 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:52.694683 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a170d9d2-c194-4b40-af59-80030fdf258f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2l5kn\" (UID: \"a170d9d2-c194-4b40-af59-80030fdf258f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l5kn" Jul 10 18:25:52.700006 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:52.698019 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a170d9d2-c194-4b40-af59-80030fdf258f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2l5kn\" (UID: \"a170d9d2-c194-4b40-af59-80030fdf258f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l5kn" Jul 10 18:25:52.953770 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:52.953670 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l5kn" Jul 10 18:25:53.121590 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:53.121558 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2l5kn"] Jul 10 18:25:53.123138 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:25:53.123102 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda170d9d2_c194_4b40_af59_80030fdf258f.slice/crio-65592f818e8d06387dfa4d691dfcfa6083d6d612d25687bd3c1b9f983e6afa9f WatchSource:0}: Error finding container 65592f818e8d06387dfa4d691dfcfa6083d6d612d25687bd3c1b9f983e6afa9f: Status 404 returned error can't find the container with id 65592f818e8d06387dfa4d691dfcfa6083d6d612d25687bd3c1b9f983e6afa9f Jul 10 18:25:53.901414 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:25:53.901373 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l5kn" event={"ID":"a170d9d2-c194-4b40-af59-80030fdf258f","Type":"ContainerStarted","Data":"65592f818e8d06387dfa4d691dfcfa6083d6d612d25687bd3c1b9f983e6afa9f"} Jul 10 18:26:00.889939 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:00.889904 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:26:02.657422 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.657384 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n"] Jul 10 18:26:02.657956 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.657605 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" podUID="4b099513-fbcd-464e-ab3f-695f58af8072" containerName="manager" containerID="cri-o://60e666a83012eae88f2deea03389b1988da418c7d5a5bb645e09e236d5d8e594" gracePeriod=2 Jul 10 18:26:02.672734 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.672699 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n"] Jul 10 18:26:02.685439 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:26:02.685393 2571 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"4b099513-fbcd-464e-ab3f-695f58af8072\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-9872n\": pods \"kuadrant-operator-controller-manager-8684c9ff49-9872n\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:26:02.688859 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.688819 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7"] Jul 10 18:26:02.689548 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.689303 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b099513-fbcd-464e-ab3f-695f58af8072" containerName="manager" Jul 10 18:26:02.689548 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.689326 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b099513-fbcd-464e-ab3f-695f58af8072" containerName="manager" Jul 10 18:26:02.689548 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.689416 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b099513-fbcd-464e-ab3f-695f58af8072" containerName="manager" Jul 10 18:26:02.701419 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.701382 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244"] Jul 10 18:26:02.701592 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.701536 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7" Jul 10 18:26:02.701747 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.701715 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244" podUID="3b79561c-7c94-400f-a1cd-7e4b3354facd" containerName="manager" containerID="cri-o://539ba0ad068a15339b539fb2ec17eb9feeccbf689f584202be8365135b39025f" gracePeriod=2 Jul 10 18:26:02.704700 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:26:02.704513 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9872n\" is forbidden: User \"system:node:ip-10-0-143-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-54.ec2.internal' and this object" podUID="4b099513-fbcd-464e-ab3f-695f58af8072" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:26:02.706919 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.706855 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244"] Jul 10 18:26:02.724128 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.723646 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7"] Jul 10 18:26:02.733928 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.733785 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zn4xq"] Jul 10 18:26:02.734636 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.734318 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b79561c-7c94-400f-a1cd-7e4b3354facd" containerName="manager" Jul 10 18:26:02.734636 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.734341 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b79561c-7c94-400f-a1cd-7e4b3354facd" containerName="manager" Jul 10 18:26:02.734636 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.734463 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b79561c-7c94-400f-a1cd-7e4b3354facd" containerName="manager" Jul 10 18:26:02.738022 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:26:02.737966 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bp244\" is forbidden: User \"system:node:ip-10-0-143-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-54.ec2.internal' and this object" podUID="3b79561c-7c94-400f-a1cd-7e4b3354facd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244" Jul 10 18:26:02.740273 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:26:02.740132 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9872n\" is forbidden: User \"system:node:ip-10-0-143-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-54.ec2.internal' and this object" podUID="4b099513-fbcd-464e-ab3f-695f58af8072" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:26:02.742192 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.742170 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zn4xq" Jul 10 18:26:02.743264 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:26:02.743225 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bp244\" is forbidden: User \"system:node:ip-10-0-143-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-54.ec2.internal' and this object" podUID="3b79561c-7c94-400f-a1cd-7e4b3354facd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244" Jul 10 18:26:02.746430 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:26:02.745602 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9872n\" is forbidden: User \"system:node:ip-10-0-143-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-54.ec2.internal' and this object" podUID="4b099513-fbcd-464e-ab3f-695f58af8072" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:26:02.748575 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:26:02.748543 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bp244\" is forbidden: User \"system:node:ip-10-0-143-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-54.ec2.internal' and this object" podUID="3b79561c-7c94-400f-a1cd-7e4b3354facd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244" Jul 10 18:26:02.748858 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.748838 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zn4xq"] Jul 10 18:26:02.790275 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.790247 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f9d8806-5157-4d40-a04e-76d60f925976-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vjcw7\" (UID: \"6f9d8806-5157-4d40-a04e-76d60f925976\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7" Jul 10 18:26:02.790457 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.790309 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgpzw\" (UniqueName: \"kubernetes.io/projected/6f9d8806-5157-4d40-a04e-76d60f925976-kube-api-access-lgpzw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vjcw7\" (UID: \"6f9d8806-5157-4d40-a04e-76d60f925976\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7" Jul 10 18:26:02.891083 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.891034 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f9d8806-5157-4d40-a04e-76d60f925976-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vjcw7\" (UID: \"6f9d8806-5157-4d40-a04e-76d60f925976\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7" Jul 10 18:26:02.891309 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.891098 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgpzw\" (UniqueName: \"kubernetes.io/projected/6f9d8806-5157-4d40-a04e-76d60f925976-kube-api-access-lgpzw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vjcw7\" (UID: \"6f9d8806-5157-4d40-a04e-76d60f925976\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7" Jul 10 18:26:02.891309 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.891135 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwfb7\" (UniqueName: \"kubernetes.io/projected/c1fc810b-3327-4e37-b42a-bb3963bafc4d-kube-api-access-dwfb7\") pod \"limitador-operator-controller-manager-859d7b78bd-zn4xq\" (UID: \"c1fc810b-3327-4e37-b42a-bb3963bafc4d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zn4xq" Jul 10 18:26:02.891577 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.891540 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f9d8806-5157-4d40-a04e-76d60f925976-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vjcw7\" (UID: \"6f9d8806-5157-4d40-a04e-76d60f925976\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7" Jul 10 18:26:02.900795 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.900764 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgpzw\" (UniqueName: \"kubernetes.io/projected/6f9d8806-5157-4d40-a04e-76d60f925976-kube-api-access-lgpzw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vjcw7\" (UID: \"6f9d8806-5157-4d40-a04e-76d60f925976\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7" Jul 10 18:26:02.992531 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:02.992488 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwfb7\" (UniqueName: \"kubernetes.io/projected/c1fc810b-3327-4e37-b42a-bb3963bafc4d-kube-api-access-dwfb7\") pod \"limitador-operator-controller-manager-859d7b78bd-zn4xq\" (UID: \"c1fc810b-3327-4e37-b42a-bb3963bafc4d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zn4xq" Jul 10 18:26:03.010001 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:03.009933 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwfb7\" (UniqueName: \"kubernetes.io/projected/c1fc810b-3327-4e37-b42a-bb3963bafc4d-kube-api-access-dwfb7\") pod \"limitador-operator-controller-manager-859d7b78bd-zn4xq\" (UID: \"c1fc810b-3327-4e37-b42a-bb3963bafc4d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zn4xq" Jul 10 18:26:03.090574 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:03.090534 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7" Jul 10 18:26:03.099806 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:03.099766 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zn4xq" Jul 10 18:26:03.682738 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:26:03.682700 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9872n\" is forbidden: User \"system:node:ip-10-0-143-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-54.ec2.internal' and this object" podUID="4b099513-fbcd-464e-ab3f-695f58af8072" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:26:03.685164 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:26:03.685125 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bp244\" is forbidden: User \"system:node:ip-10-0-143-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-54.ec2.internal' and this object" podUID="3b79561c-7c94-400f-a1cd-7e4b3354facd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244" Jul 10 18:26:17.044308 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:17.044244 2571 generic.go:356] "Generic (PLEG): container finished" podID="4b099513-fbcd-464e-ab3f-695f58af8072" containerID="60e666a83012eae88f2deea03389b1988da418c7d5a5bb645e09e236d5d8e594" exitCode=0 Jul 10 18:26:17.047835 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:17.047706 2571 generic.go:356] "Generic (PLEG): container finished" podID="3b79561c-7c94-400f-a1cd-7e4b3354facd" containerID="539ba0ad068a15339b539fb2ec17eb9feeccbf689f584202be8365135b39025f" exitCode=0 Jul 10 18:26:17.115823 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:17.115797 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244" Jul 10 18:26:17.118123 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:26:17.118064 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bp244\" is forbidden: User \"system:node:ip-10-0-143-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-54.ec2.internal' and this object" podUID="3b79561c-7c94-400f-a1cd-7e4b3354facd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244" Jul 10 18:26:17.119777 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:17.119758 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:26:17.122180 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:26:17.122150 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9872n\" is forbidden: User \"system:node:ip-10-0-143-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-54.ec2.internal' and this object" podUID="4b099513-fbcd-464e-ab3f-695f58af8072" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:26:17.124539 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:26:17.124507 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bp244\" is forbidden: User \"system:node:ip-10-0-143-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-54.ec2.internal' and this object" podUID="3b79561c-7c94-400f-a1cd-7e4b3354facd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244" Jul 10 18:26:17.129824 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:17.129802 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7"] Jul 10 18:26:17.132312 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:26:17.132287 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f9d8806_5157_4d40_a04e_76d60f925976.slice/crio-2ebe40baeb1ed9bdd1861bcb0999b49dcfe3faaf5d6d6613cdc7b32c03ae9b62 WatchSource:0}: Error finding container 2ebe40baeb1ed9bdd1861bcb0999b49dcfe3faaf5d6d6613cdc7b32c03ae9b62: Status 404 returned error can't find the container with id 2ebe40baeb1ed9bdd1861bcb0999b49dcfe3faaf5d6d6613cdc7b32c03ae9b62 Jul 10 18:26:17.151579 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:17.151539 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zn4xq"] Jul 10 18:26:17.152830 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:26:17.152797 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1fc810b_3327_4e37_b42a_bb3963bafc4d.slice/crio-d703a0ac8c60c2e92223ce68810e819c9812c7aef67e3b033e911f98fc4aad5f WatchSource:0}: Error finding container d703a0ac8c60c2e92223ce68810e819c9812c7aef67e3b033e911f98fc4aad5f: Status 404 returned error can't find the container with id d703a0ac8c60c2e92223ce68810e819c9812c7aef67e3b033e911f98fc4aad5f Jul 10 18:26:17.225645 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:17.225587 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grksr\" (UniqueName: \"kubernetes.io/projected/4b099513-fbcd-464e-ab3f-695f58af8072-kube-api-access-grksr\") pod \"4b099513-fbcd-464e-ab3f-695f58af8072\" (UID: \"4b099513-fbcd-464e-ab3f-695f58af8072\") " Jul 10 18:26:17.225837 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:17.225670 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b099513-fbcd-464e-ab3f-695f58af8072-extensions-socket-volume\") pod \"4b099513-fbcd-464e-ab3f-695f58af8072\" (UID: \"4b099513-fbcd-464e-ab3f-695f58af8072\") " Jul 10 18:26:17.225837 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:17.225715 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4r8pp\" (UniqueName: \"kubernetes.io/projected/3b79561c-7c94-400f-a1cd-7e4b3354facd-kube-api-access-4r8pp\") pod \"3b79561c-7c94-400f-a1cd-7e4b3354facd\" (UID: \"3b79561c-7c94-400f-a1cd-7e4b3354facd\") " Jul 10 18:26:17.226207 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:17.226177 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b099513-fbcd-464e-ab3f-695f58af8072-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4b099513-fbcd-464e-ab3f-695f58af8072" (UID: "4b099513-fbcd-464e-ab3f-695f58af8072"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:26:17.227848 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:17.227805 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b099513-fbcd-464e-ab3f-695f58af8072-kube-api-access-grksr" (OuterVolumeSpecName: "kube-api-access-grksr") pod "4b099513-fbcd-464e-ab3f-695f58af8072" (UID: "4b099513-fbcd-464e-ab3f-695f58af8072"). InnerVolumeSpecName "kube-api-access-grksr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:26:17.228002 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:17.227984 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b79561c-7c94-400f-a1cd-7e4b3354facd-kube-api-access-4r8pp" (OuterVolumeSpecName: "kube-api-access-4r8pp") pod "3b79561c-7c94-400f-a1cd-7e4b3354facd" (UID: "3b79561c-7c94-400f-a1cd-7e4b3354facd"). InnerVolumeSpecName "kube-api-access-4r8pp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:26:17.326567 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:17.326537 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grksr\" (UniqueName: \"kubernetes.io/projected/4b099513-fbcd-464e-ab3f-695f58af8072-kube-api-access-grksr\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:26:17.326567 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:17.326564 2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b099513-fbcd-464e-ab3f-695f58af8072-extensions-socket-volume\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:26:17.326567 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:17.326574 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r8pp\" (UniqueName: \"kubernetes.io/projected/3b79561c-7c94-400f-a1cd-7e4b3354facd-kube-api-access-4r8pp\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:26:17.683349 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:17.683304 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b79561c-7c94-400f-a1cd-7e4b3354facd" path="/var/lib/kubelet/pods/3b79561c-7c94-400f-a1cd-7e4b3354facd/volumes" Jul 10 18:26:17.683772 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:17.683752 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b099513-fbcd-464e-ab3f-695f58af8072" path="/var/lib/kubelet/pods/4b099513-fbcd-464e-ab3f-695f58af8072/volumes" Jul 10 18:26:18.053875 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:18.053847 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:26:18.053875 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:18.053861 2571 scope.go:117] "RemoveContainer" containerID="60e666a83012eae88f2deea03389b1988da418c7d5a5bb645e09e236d5d8e594" Jul 10 18:26:18.055663 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:18.055635 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l5kn" event={"ID":"a170d9d2-c194-4b40-af59-80030fdf258f","Type":"ContainerStarted","Data":"e726692336034392312fd2de319f278a6b7821d6d58077cfd9f247ae5dc8f8ce"} Jul 10 18:26:18.056302 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:26:18.056269 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9872n\" is forbidden: User \"system:node:ip-10-0-143-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-54.ec2.internal' and this object" podUID="4b099513-fbcd-464e-ab3f-695f58af8072" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:26:18.057586 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:18.057563 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zn4xq" event={"ID":"c1fc810b-3327-4e37-b42a-bb3963bafc4d","Type":"ContainerStarted","Data":"cd2c1f1a578ad75a3911e7b8634400d6ad0912ec81e7a530467a42a1e5df3e1b"} Jul 10 18:26:18.057698 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:18.057635 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zn4xq" event={"ID":"c1fc810b-3327-4e37-b42a-bb3963bafc4d","Type":"ContainerStarted","Data":"d703a0ac8c60c2e92223ce68810e819c9812c7aef67e3b033e911f98fc4aad5f"} Jul 10 18:26:18.057698 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:18.057669 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zn4xq" Jul 10 18:26:18.058513 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:26:18.058485 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9872n\" is forbidden: User \"system:node:ip-10-0-143-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-54.ec2.internal' and this object" podUID="4b099513-fbcd-464e-ab3f-695f58af8072" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:26:18.059298 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:18.059275 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7" event={"ID":"6f9d8806-5157-4d40-a04e-76d60f925976","Type":"ContainerStarted","Data":"2f7eb53f7faf5c462c9712263f895ca480d99a85fc0d812d24682e6811c58fcd"} Jul 10 18:26:18.059406 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:18.059322 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7" event={"ID":"6f9d8806-5157-4d40-a04e-76d60f925976","Type":"ContainerStarted","Data":"2ebe40baeb1ed9bdd1861bcb0999b49dcfe3faaf5d6d6613cdc7b32c03ae9b62"} Jul 10 18:26:18.059472 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:18.059454 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7" Jul 10 18:26:18.060661 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:18.060646 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244" Jul 10 18:26:18.063655 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:18.063636 2571 scope.go:117] "RemoveContainer" containerID="539ba0ad068a15339b539fb2ec17eb9feeccbf689f584202be8365135b39025f" Jul 10 18:26:18.076170 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:18.076124 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2l5kn" podStartSLOduration=2.191346323 podStartE2EDuration="26.07610845s" totalImagesPullingTime="23.884762127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:25:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:26:18.075155835 +0000 UTC m=+534.974424944" watchObservedRunningTime="2026-07-10 18:26:18.07610845 +0000 UTC m=+534.975377557" Jul 10 18:26:18.077292 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:26:18.077271 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9872n\" is forbidden: User \"system:node:ip-10-0-143-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-54.ec2.internal' and this object" podUID="4b099513-fbcd-464e-ab3f-695f58af8072" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:26:18.101216 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:18.101159 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7" podStartSLOduration=16.101140521 podStartE2EDuration="16.101140521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:26:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:26:18.099732832 +0000 UTC m=+534.999001972" watchObservedRunningTime="2026-07-10 18:26:18.101140521 +0000 UTC m=+535.000409629" Jul 10 18:26:18.101917 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:26:18.101885 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bp244\" is forbidden: User \"system:node:ip-10-0-143-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-54.ec2.internal' and this object" podUID="3b79561c-7c94-400f-a1cd-7e4b3354facd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244" Jul 10 18:26:18.124191 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:18.124125 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zn4xq" podStartSLOduration=16.124101211 podStartE2EDuration="16.124101211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:26:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:26:18.120736881 +0000 UTC m=+535.020005999" watchObservedRunningTime="2026-07-10 18:26:18.124101211 +0000 UTC m=+535.023370320" Jul 10 18:26:18.559940 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:18.559910 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7"] Jul 10 18:26:18.801935 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:18.801895 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f"] Jul 10 18:26:18.805113 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:18.805091 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f" Jul 10 18:26:18.822879 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:18.822847 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f"] Jul 10 18:26:18.846806 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:26:18.846765 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9872n\" is forbidden: User \"system:node:ip-10-0-143-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-54.ec2.internal' and this object" podUID="4b099513-fbcd-464e-ab3f-695f58af8072" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9872n" Jul 10 18:26:18.856758 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:26:18.856726 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bp244\" is forbidden: User \"system:node:ip-10-0-143-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-54.ec2.internal' and this object" podUID="3b79561c-7c94-400f-a1cd-7e4b3354facd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bp244" Jul 10 18:26:18.940063 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:18.940022 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed995b0f-5598-4cb9-a0f9-5f84d083cb3b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bnd7f\" (UID: \"ed995b0f-5598-4cb9-a0f9-5f84d083cb3b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f" Jul 10 18:26:18.940259 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:18.940081 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc92c\" (UniqueName: \"kubernetes.io/projected/ed995b0f-5598-4cb9-a0f9-5f84d083cb3b-kube-api-access-vc92c\") pod \"kuadrant-operator-controller-manager-86878c769f-bnd7f\" (UID: \"ed995b0f-5598-4cb9-a0f9-5f84d083cb3b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f" Jul 10 18:26:19.041579 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:19.041522 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc92c\" (UniqueName: \"kubernetes.io/projected/ed995b0f-5598-4cb9-a0f9-5f84d083cb3b-kube-api-access-vc92c\") pod \"kuadrant-operator-controller-manager-86878c769f-bnd7f\" (UID: \"ed995b0f-5598-4cb9-a0f9-5f84d083cb3b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f" Jul 10 18:26:19.041753 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:19.041632 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed995b0f-5598-4cb9-a0f9-5f84d083cb3b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bnd7f\" (UID: \"ed995b0f-5598-4cb9-a0f9-5f84d083cb3b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f" Jul 10 18:26:19.041987 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:19.041970 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed995b0f-5598-4cb9-a0f9-5f84d083cb3b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bnd7f\" (UID: \"ed995b0f-5598-4cb9-a0f9-5f84d083cb3b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f" Jul 10 18:26:19.053374 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:19.053341 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc92c\" (UniqueName: \"kubernetes.io/projected/ed995b0f-5598-4cb9-a0f9-5f84d083cb3b-kube-api-access-vc92c\") pod \"kuadrant-operator-controller-manager-86878c769f-bnd7f\" (UID: \"ed995b0f-5598-4cb9-a0f9-5f84d083cb3b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f" Jul 10 18:26:19.115521 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:19.115427 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f" Jul 10 18:26:19.260596 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:19.260571 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f"] Jul 10 18:26:19.262896 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:26:19.262870 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded995b0f_5598_4cb9_a0f9_5f84d083cb3b.slice/crio-42e35e37be57ec26552d0da9dcbfc4f6d543991d3db8671f6d743a903f5d08b3 WatchSource:0}: Error finding container 42e35e37be57ec26552d0da9dcbfc4f6d543991d3db8671f6d743a903f5d08b3: Status 404 returned error can't find the container with id 42e35e37be57ec26552d0da9dcbfc4f6d543991d3db8671f6d743a903f5d08b3 Jul 10 18:26:20.073921 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:20.073879 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f" event={"ID":"ed995b0f-5598-4cb9-a0f9-5f84d083cb3b","Type":"ContainerStarted","Data":"d3bd6d11a4e5c6d366a93a16f0e706e8faf44192ce7d3378b4a8120756c18ab5"} Jul 10 18:26:20.074084 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:20.073961 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f" event={"ID":"ed995b0f-5598-4cb9-a0f9-5f84d083cb3b","Type":"ContainerStarted","Data":"42e35e37be57ec26552d0da9dcbfc4f6d543991d3db8671f6d743a903f5d08b3"} Jul 10 18:26:20.074084 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:20.074008 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7" podUID="6f9d8806-5157-4d40-a04e-76d60f925976" containerName="manager" containerID="cri-o://2f7eb53f7faf5c462c9712263f895ca480d99a85fc0d812d24682e6811c58fcd" gracePeriod=10 Jul 10 18:26:20.097043 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:20.096995 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f" podStartSLOduration=2.096980081 podStartE2EDuration="2.096980081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:26:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:26:20.095508838 +0000 UTC m=+536.994777946" watchObservedRunningTime="2026-07-10 18:26:20.096980081 +0000 UTC m=+536.996249189" Jul 10 18:26:20.324666 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:20.324585 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7" Jul 10 18:26:20.453337 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:20.453298 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f9d8806-5157-4d40-a04e-76d60f925976-extensions-socket-volume\") pod \"6f9d8806-5157-4d40-a04e-76d60f925976\" (UID: \"6f9d8806-5157-4d40-a04e-76d60f925976\") " Jul 10 18:26:20.453534 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:20.453355 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgpzw\" (UniqueName: \"kubernetes.io/projected/6f9d8806-5157-4d40-a04e-76d60f925976-kube-api-access-lgpzw\") pod \"6f9d8806-5157-4d40-a04e-76d60f925976\" (UID: \"6f9d8806-5157-4d40-a04e-76d60f925976\") " Jul 10 18:26:20.453744 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:20.453715 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f9d8806-5157-4d40-a04e-76d60f925976-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6f9d8806-5157-4d40-a04e-76d60f925976" (UID: "6f9d8806-5157-4d40-a04e-76d60f925976"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:26:20.455508 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:20.455482 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f9d8806-5157-4d40-a04e-76d60f925976-kube-api-access-lgpzw" (OuterVolumeSpecName: "kube-api-access-lgpzw") pod "6f9d8806-5157-4d40-a04e-76d60f925976" (UID: "6f9d8806-5157-4d40-a04e-76d60f925976"). InnerVolumeSpecName "kube-api-access-lgpzw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:26:20.554847 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:20.554801 2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f9d8806-5157-4d40-a04e-76d60f925976-extensions-socket-volume\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:26:20.554847 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:20.554839 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgpzw\" (UniqueName: \"kubernetes.io/projected/6f9d8806-5157-4d40-a04e-76d60f925976-kube-api-access-lgpzw\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:26:21.080839 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:21.080802 2571 generic.go:356] "Generic (PLEG): container finished" podID="6f9d8806-5157-4d40-a04e-76d60f925976" containerID="2f7eb53f7faf5c462c9712263f895ca480d99a85fc0d812d24682e6811c58fcd" exitCode=0 Jul 10 18:26:21.081026 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:21.080857 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7" Jul 10 18:26:21.081026 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:21.080882 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7" event={"ID":"6f9d8806-5157-4d40-a04e-76d60f925976","Type":"ContainerDied","Data":"2f7eb53f7faf5c462c9712263f895ca480d99a85fc0d812d24682e6811c58fcd"} Jul 10 18:26:21.081026 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:21.080932 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7" event={"ID":"6f9d8806-5157-4d40-a04e-76d60f925976","Type":"ContainerDied","Data":"2ebe40baeb1ed9bdd1861bcb0999b49dcfe3faaf5d6d6613cdc7b32c03ae9b62"} Jul 10 18:26:21.081026 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:21.080956 2571 scope.go:117] "RemoveContainer" containerID="2f7eb53f7faf5c462c9712263f895ca480d99a85fc0d812d24682e6811c58fcd" Jul 10 18:26:21.081362 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:21.081346 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f" Jul 10 18:26:21.090045 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:21.090028 2571 scope.go:117] "RemoveContainer" containerID="2f7eb53f7faf5c462c9712263f895ca480d99a85fc0d812d24682e6811c58fcd" Jul 10 18:26:21.090313 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:26:21.090289 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f7eb53f7faf5c462c9712263f895ca480d99a85fc0d812d24682e6811c58fcd\": container with ID starting with 2f7eb53f7faf5c462c9712263f895ca480d99a85fc0d812d24682e6811c58fcd not found: ID does not exist" containerID="2f7eb53f7faf5c462c9712263f895ca480d99a85fc0d812d24682e6811c58fcd" Jul 10 18:26:21.090362 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:21.090326 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f7eb53f7faf5c462c9712263f895ca480d99a85fc0d812d24682e6811c58fcd"} err="failed to get container status \"2f7eb53f7faf5c462c9712263f895ca480d99a85fc0d812d24682e6811c58fcd\": rpc error: code = NotFound desc = could not find container \"2f7eb53f7faf5c462c9712263f895ca480d99a85fc0d812d24682e6811c58fcd\": container with ID starting with 2f7eb53f7faf5c462c9712263f895ca480d99a85fc0d812d24682e6811c58fcd not found: ID does not exist" Jul 10 18:26:21.106226 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:21.106196 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7"] Jul 10 18:26:21.110242 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:21.110219 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vjcw7"] Jul 10 18:26:21.683503 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:21.683468 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f9d8806-5157-4d40-a04e-76d60f925976" path="/var/lib/kubelet/pods/6f9d8806-5157-4d40-a04e-76d60f925976/volumes" Jul 10 18:26:29.070388 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:29.070361 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zn4xq" Jul 10 18:26:32.089839 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:32.089797 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f" Jul 10 18:26:35.002542 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.002500 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz"] Jul 10 18:26:35.003006 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.002907 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f9d8806-5157-4d40-a04e-76d60f925976" containerName="manager" Jul 10 18:26:35.003006 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.002921 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f9d8806-5157-4d40-a04e-76d60f925976" containerName="manager" Jul 10 18:26:35.003006 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.002985 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f9d8806-5157-4d40-a04e-76d60f925976" containerName="manager" Jul 10 18:26:35.021633 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.021579 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz"] Jul 10 18:26:35.021791 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.021735 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.024648 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.024601 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-gm2kc\"" Jul 10 18:26:35.070453 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.070413 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31a095f9-f006-476e-b42b-d7682d0f463b-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.070453 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.070458 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31a095f9-f006-476e-b42b-d7682d0f463b-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.070723 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.070518 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd6zt\" (UniqueName: \"kubernetes.io/projected/31a095f9-f006-476e-b42b-d7682d0f463b-kube-api-access-kd6zt\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.070723 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.070538 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31a095f9-f006-476e-b42b-d7682d0f463b-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.070723 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.070563 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31a095f9-f006-476e-b42b-d7682d0f463b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.070723 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.070590 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31a095f9-f006-476e-b42b-d7682d0f463b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.070723 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.070633 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31a095f9-f006-476e-b42b-d7682d0f463b-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.070723 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.070684 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31a095f9-f006-476e-b42b-d7682d0f463b-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.070723 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.070718 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31a095f9-f006-476e-b42b-d7682d0f463b-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.171534 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.171491 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31a095f9-f006-476e-b42b-d7682d0f463b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.171534 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.171538 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31a095f9-f006-476e-b42b-d7682d0f463b-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.171815 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.171569 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31a095f9-f006-476e-b42b-d7682d0f463b-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.171815 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.171594 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31a095f9-f006-476e-b42b-d7682d0f463b-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.171815 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.171662 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31a095f9-f006-476e-b42b-d7682d0f463b-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.171815 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.171701 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31a095f9-f006-476e-b42b-d7682d0f463b-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.171815 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.171752 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd6zt\" (UniqueName: \"kubernetes.io/projected/31a095f9-f006-476e-b42b-d7682d0f463b-kube-api-access-kd6zt\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.171815 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.171776 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31a095f9-f006-476e-b42b-d7682d0f463b-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.171815 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.171800 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31a095f9-f006-476e-b42b-d7682d0f463b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.172133 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.172048 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31a095f9-f006-476e-b42b-d7682d0f463b-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.172191 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.172137 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31a095f9-f006-476e-b42b-d7682d0f463b-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.172191 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.172171 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31a095f9-f006-476e-b42b-d7682d0f463b-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.172415 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.172390 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31a095f9-f006-476e-b42b-d7682d0f463b-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.172488 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.172417 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31a095f9-f006-476e-b42b-d7682d0f463b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.173934 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.173905 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31a095f9-f006-476e-b42b-d7682d0f463b-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.174259 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.174239 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31a095f9-f006-476e-b42b-d7682d0f463b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.181143 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.181118 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31a095f9-f006-476e-b42b-d7682d0f463b-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.181263 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.181239 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd6zt\" (UniqueName: \"kubernetes.io/projected/31a095f9-f006-476e-b42b-d7682d0f463b-kube-api-access-kd6zt\") pod \"maas-default-gateway-openshift-default-5896d4dd49-msnmz\" (UID: \"31a095f9-f006-476e-b42b-d7682d0f463b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.335220 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.335132 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:35.484600 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.484570 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz"] Jul 10 18:26:35.486710 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:26:35.486680 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31a095f9_f006_476e_b42b_d7682d0f463b.slice/crio-8b872a2dafcdaa8eae10143950efdb8bbac92c805f0ae54dc2f595e5e5b9708d WatchSource:0}: Error finding container 8b872a2dafcdaa8eae10143950efdb8bbac92c805f0ae54dc2f595e5e5b9708d: Status 404 returned error can't find the container with id 8b872a2dafcdaa8eae10143950efdb8bbac92c805f0ae54dc2f595e5e5b9708d Jul 10 18:26:35.488917 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.488885 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:26:35.488996 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.488955 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:26:35.488996 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:35.488987 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:26:36.155980 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:36.155945 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" event={"ID":"31a095f9-f006-476e-b42b-d7682d0f463b","Type":"ContainerStarted","Data":"6d9cfec587c4fa52a230afc078832fab909d2fd3c576f93eba84179c5e29d56b"} Jul 10 18:26:36.156388 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:36.155997 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" event={"ID":"31a095f9-f006-476e-b42b-d7682d0f463b","Type":"ContainerStarted","Data":"8b872a2dafcdaa8eae10143950efdb8bbac92c805f0ae54dc2f595e5e5b9708d"} Jul 10 18:26:36.178523 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:36.178466 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" podStartSLOduration=2.178446761 podStartE2EDuration="2.178446761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:26:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:26:36.176915353 +0000 UTC m=+553.076184505" watchObservedRunningTime="2026-07-10 18:26:36.178446761 +0000 UTC m=+553.077715868" Jul 10 18:26:36.335405 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:36.335370 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:36.340787 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:36.340759 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:37.161635 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:37.161591 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:37.162751 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:37.162731 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:26:53.792863 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:53.792825 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-jj7r5"] Jul 10 18:26:53.796233 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:53.796208 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-jj7r5" Jul 10 18:26:53.798956 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:53.798931 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bd7xg\"" Jul 10 18:26:53.807683 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:53.807653 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-jj7r5"] Jul 10 18:26:53.936674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:53.936629 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5vtf\" (UniqueName: \"kubernetes.io/projected/acc37ee0-a4c0-497c-b8e2-c8338d0e5399-kube-api-access-b5vtf\") pod \"authorino-58cd955675-jj7r5\" (UID: \"acc37ee0-a4c0-497c-b8e2-c8338d0e5399\") " pod="kuadrant-system/authorino-58cd955675-jj7r5" Jul 10 18:26:54.034910 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:54.034874 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-4hvqc"] Jul 10 18:26:54.037765 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:54.037728 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4hvqc" Jul 10 18:26:54.037971 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:54.037950 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5vtf\" (UniqueName: \"kubernetes.io/projected/acc37ee0-a4c0-497c-b8e2-c8338d0e5399-kube-api-access-b5vtf\") pod \"authorino-58cd955675-jj7r5\" (UID: \"acc37ee0-a4c0-497c-b8e2-c8338d0e5399\") " pod="kuadrant-system/authorino-58cd955675-jj7r5" Jul 10 18:26:54.046413 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:54.046380 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4hvqc"] Jul 10 18:26:54.048202 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:54.048171 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5vtf\" (UniqueName: \"kubernetes.io/projected/acc37ee0-a4c0-497c-b8e2-c8338d0e5399-kube-api-access-b5vtf\") pod \"authorino-58cd955675-jj7r5\" (UID: \"acc37ee0-a4c0-497c-b8e2-c8338d0e5399\") " pod="kuadrant-system/authorino-58cd955675-jj7r5" Jul 10 18:26:54.106834 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:54.106800 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-jj7r5" Jul 10 18:26:54.138564 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:54.138528 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tksj\" (UniqueName: \"kubernetes.io/projected/7ca1438a-25eb-4c11-a17d-655b18c4370b-kube-api-access-2tksj\") pod \"authorino-75d556bd77-4hvqc\" (UID: \"7ca1438a-25eb-4c11-a17d-655b18c4370b\") " pod="kuadrant-system/authorino-75d556bd77-4hvqc" Jul 10 18:26:54.239314 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:54.239277 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tksj\" (UniqueName: \"kubernetes.io/projected/7ca1438a-25eb-4c11-a17d-655b18c4370b-kube-api-access-2tksj\") pod \"authorino-75d556bd77-4hvqc\" (UID: \"7ca1438a-25eb-4c11-a17d-655b18c4370b\") " pod="kuadrant-system/authorino-75d556bd77-4hvqc" Jul 10 18:26:54.244402 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:54.244380 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-jj7r5"] Jul 10 18:26:54.246728 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:26:54.246686 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podacc37ee0_a4c0_497c_b8e2_c8338d0e5399.slice/crio-c34ef8713bb892a8bf48db5f70d3b40b75ed7ba9bbfe58c67abecb94c5681579 WatchSource:0}: Error finding container c34ef8713bb892a8bf48db5f70d3b40b75ed7ba9bbfe58c67abecb94c5681579: Status 404 returned error can't find the container with id c34ef8713bb892a8bf48db5f70d3b40b75ed7ba9bbfe58c67abecb94c5681579 Jul 10 18:26:54.250086 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:54.250063 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tksj\" (UniqueName: \"kubernetes.io/projected/7ca1438a-25eb-4c11-a17d-655b18c4370b-kube-api-access-2tksj\") pod \"authorino-75d556bd77-4hvqc\" (UID: \"7ca1438a-25eb-4c11-a17d-655b18c4370b\") " pod="kuadrant-system/authorino-75d556bd77-4hvqc" Jul 10 18:26:54.348668 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:54.348543 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4hvqc" Jul 10 18:26:54.486366 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:54.486340 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4hvqc"] Jul 10 18:26:54.488358 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:26:54.488336 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ca1438a_25eb_4c11_a17d_655b18c4370b.slice/crio-dabf80cda296f703c7db5219c5f7ebdea32222f6ec6c49d1e1bda987a1f42386 WatchSource:0}: Error finding container dabf80cda296f703c7db5219c5f7ebdea32222f6ec6c49d1e1bda987a1f42386: Status 404 returned error can't find the container with id dabf80cda296f703c7db5219c5f7ebdea32222f6ec6c49d1e1bda987a1f42386 Jul 10 18:26:55.259022 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:55.258960 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4hvqc" event={"ID":"7ca1438a-25eb-4c11-a17d-655b18c4370b","Type":"ContainerStarted","Data":"dabf80cda296f703c7db5219c5f7ebdea32222f6ec6c49d1e1bda987a1f42386"} Jul 10 18:26:55.265116 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:55.265079 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-jj7r5" event={"ID":"acc37ee0-a4c0-497c-b8e2-c8338d0e5399","Type":"ContainerStarted","Data":"c34ef8713bb892a8bf48db5f70d3b40b75ed7ba9bbfe58c67abecb94c5681579"} Jul 10 18:26:58.283465 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:58.283430 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-jj7r5" event={"ID":"acc37ee0-a4c0-497c-b8e2-c8338d0e5399","Type":"ContainerStarted","Data":"5327e6a3a5d4df2edf105984592f104a999dc662af966f9f1b555b9e60bc9c20"} Jul 10 18:26:58.284956 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:58.284935 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4hvqc" event={"ID":"7ca1438a-25eb-4c11-a17d-655b18c4370b","Type":"ContainerStarted","Data":"b95e976f9e762a8e1682fd7d4f4950745bafdb4aeb02353304724515d3e669b0"} Jul 10 18:26:58.301499 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:58.301449 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-jj7r5" podStartSLOduration=1.7511582319999999 podStartE2EDuration="5.301433213s" totalImagesPullingTime="3.550274981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:26:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:26:58.300336884 +0000 UTC m=+575.199606015" watchObservedRunningTime="2026-07-10 18:26:58.301433213 +0000 UTC m=+575.200702321" Jul 10 18:26:58.319724 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:58.319671 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-4hvqc" podStartSLOduration=0.998293855 podStartE2EDuration="4.319651947s" totalImagesPullingTime="3.321358092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:26:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:26:58.318977353 +0000 UTC m=+575.218246462" watchObservedRunningTime="2026-07-10 18:26:58.319651947 +0000 UTC m=+575.218921051" Jul 10 18:26:58.350444 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:26:58.350405 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-jj7r5"] Jul 10 18:27:00.299949 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:00.299882 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-jj7r5" podUID="acc37ee0-a4c0-497c-b8e2-c8338d0e5399" containerName="authorino" containerID="cri-o://5327e6a3a5d4df2edf105984592f104a999dc662af966f9f1b555b9e60bc9c20" gracePeriod=30 Jul 10 18:27:00.552370 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:00.552309 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-jj7r5" Jul 10 18:27:00.700504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:00.700467 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5vtf\" (UniqueName: \"kubernetes.io/projected/acc37ee0-a4c0-497c-b8e2-c8338d0e5399-kube-api-access-b5vtf\") pod \"acc37ee0-a4c0-497c-b8e2-c8338d0e5399\" (UID: \"acc37ee0-a4c0-497c-b8e2-c8338d0e5399\") " Jul 10 18:27:00.702547 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:00.702522 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acc37ee0-a4c0-497c-b8e2-c8338d0e5399-kube-api-access-b5vtf" (OuterVolumeSpecName: "kube-api-access-b5vtf") pod "acc37ee0-a4c0-497c-b8e2-c8338d0e5399" (UID: "acc37ee0-a4c0-497c-b8e2-c8338d0e5399"). InnerVolumeSpecName "kube-api-access-b5vtf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:27:00.802117 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:00.802084 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5vtf\" (UniqueName: \"kubernetes.io/projected/acc37ee0-a4c0-497c-b8e2-c8338d0e5399-kube-api-access-b5vtf\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:27:01.305511 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:01.305477 2571 generic.go:356] "Generic (PLEG): container finished" podID="acc37ee0-a4c0-497c-b8e2-c8338d0e5399" containerID="5327e6a3a5d4df2edf105984592f104a999dc662af966f9f1b555b9e60bc9c20" exitCode=0 Jul 10 18:27:01.306007 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:01.305521 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-jj7r5" Jul 10 18:27:01.306007 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:01.305551 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-jj7r5" event={"ID":"acc37ee0-a4c0-497c-b8e2-c8338d0e5399","Type":"ContainerDied","Data":"5327e6a3a5d4df2edf105984592f104a999dc662af966f9f1b555b9e60bc9c20"} Jul 10 18:27:01.306007 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:01.305584 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-jj7r5" event={"ID":"acc37ee0-a4c0-497c-b8e2-c8338d0e5399","Type":"ContainerDied","Data":"c34ef8713bb892a8bf48db5f70d3b40b75ed7ba9bbfe58c67abecb94c5681579"} Jul 10 18:27:01.306007 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:01.305600 2571 scope.go:117] "RemoveContainer" containerID="5327e6a3a5d4df2edf105984592f104a999dc662af966f9f1b555b9e60bc9c20" Jul 10 18:27:01.314538 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:01.314521 2571 scope.go:117] "RemoveContainer" containerID="5327e6a3a5d4df2edf105984592f104a999dc662af966f9f1b555b9e60bc9c20" Jul 10 18:27:01.314828 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:27:01.314812 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5327e6a3a5d4df2edf105984592f104a999dc662af966f9f1b555b9e60bc9c20\": container with ID starting with 5327e6a3a5d4df2edf105984592f104a999dc662af966f9f1b555b9e60bc9c20 not found: ID does not exist" containerID="5327e6a3a5d4df2edf105984592f104a999dc662af966f9f1b555b9e60bc9c20" Jul 10 18:27:01.314889 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:01.314838 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5327e6a3a5d4df2edf105984592f104a999dc662af966f9f1b555b9e60bc9c20"} err="failed to get container status \"5327e6a3a5d4df2edf105984592f104a999dc662af966f9f1b555b9e60bc9c20\": rpc error: code = NotFound desc = could not find container \"5327e6a3a5d4df2edf105984592f104a999dc662af966f9f1b555b9e60bc9c20\": container with ID starting with 5327e6a3a5d4df2edf105984592f104a999dc662af966f9f1b555b9e60bc9c20 not found: ID does not exist" Jul 10 18:27:01.328029 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:01.327993 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-jj7r5"] Jul 10 18:27:01.332879 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:01.332851 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-jj7r5"] Jul 10 18:27:01.683396 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:01.683317 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acc37ee0-a4c0-497c-b8e2-c8338d0e5399" path="/var/lib/kubelet/pods/acc37ee0-a4c0-497c-b8e2-c8338d0e5399/volumes" Jul 10 18:27:23.527312 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:23.527285 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r9vz_0cafad48-3998-465d-b4c8-b5d94cac6b3c/ovn-acl-logging/0.log" Jul 10 18:27:23.527871 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:23.527842 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r9vz_0cafad48-3998-465d-b4c8-b5d94cac6b3c/ovn-acl-logging/0.log" Jul 10 18:27:26.424439 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.424407 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ptpd9"] Jul 10 18:27:26.424884 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.424869 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acc37ee0-a4c0-497c-b8e2-c8338d0e5399" containerName="authorino" Jul 10 18:27:26.424971 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.424890 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="acc37ee0-a4c0-497c-b8e2-c8338d0e5399" containerName="authorino" Jul 10 18:27:26.425037 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.424992 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="acc37ee0-a4c0-497c-b8e2-c8338d0e5399" containerName="authorino" Jul 10 18:27:26.429537 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.429515 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ptpd9" Jul 10 18:27:26.435229 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.435205 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ptpd9"] Jul 10 18:27:26.515673 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.515608 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bchl2\" (UniqueName: \"kubernetes.io/projected/58bac243-2634-4106-a98f-66718966a98a-kube-api-access-bchl2\") pod \"authorino-5d4b67c446-ptpd9\" (UID: \"58bac243-2634-4106-a98f-66718966a98a\") " pod="kuadrant-system/authorino-5d4b67c446-ptpd9" Jul 10 18:27:26.616230 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.616173 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bchl2\" (UniqueName: \"kubernetes.io/projected/58bac243-2634-4106-a98f-66718966a98a-kube-api-access-bchl2\") pod \"authorino-5d4b67c446-ptpd9\" (UID: \"58bac243-2634-4106-a98f-66718966a98a\") " pod="kuadrant-system/authorino-5d4b67c446-ptpd9" Jul 10 18:27:26.624441 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.624407 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bchl2\" (UniqueName: \"kubernetes.io/projected/58bac243-2634-4106-a98f-66718966a98a-kube-api-access-bchl2\") pod \"authorino-5d4b67c446-ptpd9\" (UID: \"58bac243-2634-4106-a98f-66718966a98a\") " pod="kuadrant-system/authorino-5d4b67c446-ptpd9" Jul 10 18:27:26.662892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.662858 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ptpd9"] Jul 10 18:27:26.663117 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.663105 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ptpd9" Jul 10 18:27:26.692488 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.692401 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-649dd84959-lslz5"] Jul 10 18:27:26.697440 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.697413 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-649dd84959-lslz5" Jul 10 18:27:26.702080 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.702049 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-649dd84959-lslz5"] Jul 10 18:27:26.717113 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.717083 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpq4f\" (UniqueName: \"kubernetes.io/projected/d8f30189-86c9-4feb-bc82-6410f891f5ff-kube-api-access-xpq4f\") pod \"authorino-649dd84959-lslz5\" (UID: \"d8f30189-86c9-4feb-bc82-6410f891f5ff\") " pod="kuadrant-system/authorino-649dd84959-lslz5" Jul 10 18:27:26.768963 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.768920 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-649dd84959-lslz5"] Jul 10 18:27:26.769213 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:27:26.769193 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-xpq4f], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-649dd84959-lslz5" podUID="d8f30189-86c9-4feb-bc82-6410f891f5ff" Jul 10 18:27:26.802503 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.802471 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-54c6b9cfb7-jvsxq"] Jul 10 18:27:26.805834 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.805816 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54c6b9cfb7-jvsxq" Jul 10 18:27:26.808651 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.808626 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 10 18:27:26.810901 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.810871 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ptpd9"] Jul 10 18:27:26.813668 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:27:26.813644 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58bac243_2634_4106_a98f_66718966a98a.slice/crio-40b0b7c8f29ea89757c362cbcccfa1bf83d62dba4304740e1f4cbabf0365719b WatchSource:0}: Error finding container 40b0b7c8f29ea89757c362cbcccfa1bf83d62dba4304740e1f4cbabf0365719b: Status 404 returned error can't find the container with id 40b0b7c8f29ea89757c362cbcccfa1bf83d62dba4304740e1f4cbabf0365719b Jul 10 18:27:26.813972 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.813957 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54c6b9cfb7-jvsxq"] Jul 10 18:27:26.817695 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.817670 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzxz9\" (UniqueName: \"kubernetes.io/projected/de834122-1105-4eb3-8693-4564ba44d3cd-kube-api-access-gzxz9\") pod \"authorino-54c6b9cfb7-jvsxq\" (UID: \"de834122-1105-4eb3-8693-4564ba44d3cd\") " pod="kuadrant-system/authorino-54c6b9cfb7-jvsxq" Jul 10 18:27:26.817807 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.817719 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpq4f\" (UniqueName: \"kubernetes.io/projected/d8f30189-86c9-4feb-bc82-6410f891f5ff-kube-api-access-xpq4f\") pod \"authorino-649dd84959-lslz5\" (UID: \"d8f30189-86c9-4feb-bc82-6410f891f5ff\") " pod="kuadrant-system/authorino-649dd84959-lslz5" Jul 10 18:27:26.817807 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.817749 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/de834122-1105-4eb3-8693-4564ba44d3cd-tls-cert\") pod \"authorino-54c6b9cfb7-jvsxq\" (UID: \"de834122-1105-4eb3-8693-4564ba44d3cd\") " pod="kuadrant-system/authorino-54c6b9cfb7-jvsxq" Jul 10 18:27:26.829312 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.829282 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpq4f\" (UniqueName: \"kubernetes.io/projected/d8f30189-86c9-4feb-bc82-6410f891f5ff-kube-api-access-xpq4f\") pod \"authorino-649dd84959-lslz5\" (UID: \"d8f30189-86c9-4feb-bc82-6410f891f5ff\") " pod="kuadrant-system/authorino-649dd84959-lslz5" Jul 10 18:27:26.918545 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.918490 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzxz9\" (UniqueName: \"kubernetes.io/projected/de834122-1105-4eb3-8693-4564ba44d3cd-kube-api-access-gzxz9\") pod \"authorino-54c6b9cfb7-jvsxq\" (UID: \"de834122-1105-4eb3-8693-4564ba44d3cd\") " pod="kuadrant-system/authorino-54c6b9cfb7-jvsxq" Jul 10 18:27:26.918545 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.918554 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/de834122-1105-4eb3-8693-4564ba44d3cd-tls-cert\") pod \"authorino-54c6b9cfb7-jvsxq\" (UID: \"de834122-1105-4eb3-8693-4564ba44d3cd\") " pod="kuadrant-system/authorino-54c6b9cfb7-jvsxq" Jul 10 18:27:26.921014 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.920992 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/de834122-1105-4eb3-8693-4564ba44d3cd-tls-cert\") pod \"authorino-54c6b9cfb7-jvsxq\" (UID: \"de834122-1105-4eb3-8693-4564ba44d3cd\") " pod="kuadrant-system/authorino-54c6b9cfb7-jvsxq" Jul 10 18:27:26.926902 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:26.926879 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzxz9\" (UniqueName: \"kubernetes.io/projected/de834122-1105-4eb3-8693-4564ba44d3cd-kube-api-access-gzxz9\") pod \"authorino-54c6b9cfb7-jvsxq\" (UID: \"de834122-1105-4eb3-8693-4564ba44d3cd\") " pod="kuadrant-system/authorino-54c6b9cfb7-jvsxq" Jul 10 18:27:27.125579 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:27.125539 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54c6b9cfb7-jvsxq" Jul 10 18:27:27.279922 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:27.279897 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54c6b9cfb7-jvsxq"] Jul 10 18:27:27.282281 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:27:27.282254 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde834122_1105_4eb3_8693_4564ba44d3cd.slice/crio-065a0cf9904ab771baa1bf1e6c47f239bbacab5bfc2b1241eb1380af8ab25e14 WatchSource:0}: Error finding container 065a0cf9904ab771baa1bf1e6c47f239bbacab5bfc2b1241eb1380af8ab25e14: Status 404 returned error can't find the container with id 065a0cf9904ab771baa1bf1e6c47f239bbacab5bfc2b1241eb1380af8ab25e14 Jul 10 18:27:27.435695 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:27.435575 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-ptpd9" event={"ID":"58bac243-2634-4106-a98f-66718966a98a","Type":"ContainerStarted","Data":"fbf35d4f4264cb143f6c01cef44772851400a80c59b6eeba9ee9fa775a899b2b"} Jul 10 18:27:27.435695 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:27.435642 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-ptpd9" event={"ID":"58bac243-2634-4106-a98f-66718966a98a","Type":"ContainerStarted","Data":"40b0b7c8f29ea89757c362cbcccfa1bf83d62dba4304740e1f4cbabf0365719b"} Jul 10 18:27:27.435695 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:27.435665 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-ptpd9" podUID="58bac243-2634-4106-a98f-66718966a98a" containerName="authorino" containerID="cri-o://fbf35d4f4264cb143f6c01cef44772851400a80c59b6eeba9ee9fa775a899b2b" gracePeriod=30 Jul 10 18:27:27.436901 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:27.436878 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54c6b9cfb7-jvsxq" event={"ID":"de834122-1105-4eb3-8693-4564ba44d3cd","Type":"ContainerStarted","Data":"065a0cf9904ab771baa1bf1e6c47f239bbacab5bfc2b1241eb1380af8ab25e14"} Jul 10 18:27:27.436901 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:27.436896 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-649dd84959-lslz5" Jul 10 18:27:27.449889 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:27.449859 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-649dd84959-lslz5" Jul 10 18:27:27.450535 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:27.450495 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-ptpd9" podStartSLOduration=1.092844769 podStartE2EDuration="1.450482685s" totalImagesPullingTime="357.637916ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:27:27.449719742 +0000 UTC m=+604.348988850" watchObservedRunningTime="2026-07-10 18:27:27.450482685 +0000 UTC m=+604.349751793" Jul 10 18:27:27.523776 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:27.523740 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpq4f\" (UniqueName: \"kubernetes.io/projected/d8f30189-86c9-4feb-bc82-6410f891f5ff-kube-api-access-xpq4f\") pod \"d8f30189-86c9-4feb-bc82-6410f891f5ff\" (UID: \"d8f30189-86c9-4feb-bc82-6410f891f5ff\") " Jul 10 18:27:27.525840 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:27.525812 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8f30189-86c9-4feb-bc82-6410f891f5ff-kube-api-access-xpq4f" (OuterVolumeSpecName: "kube-api-access-xpq4f") pod "d8f30189-86c9-4feb-bc82-6410f891f5ff" (UID: "d8f30189-86c9-4feb-bc82-6410f891f5ff"). InnerVolumeSpecName "kube-api-access-xpq4f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:27:27.624333 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:27.624296 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpq4f\" (UniqueName: \"kubernetes.io/projected/d8f30189-86c9-4feb-bc82-6410f891f5ff-kube-api-access-xpq4f\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:27:27.702122 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:27.702097 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ptpd9" Jul 10 18:27:27.725177 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:27.725144 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bchl2\" (UniqueName: \"kubernetes.io/projected/58bac243-2634-4106-a98f-66718966a98a-kube-api-access-bchl2\") pod \"58bac243-2634-4106-a98f-66718966a98a\" (UID: \"58bac243-2634-4106-a98f-66718966a98a\") " Jul 10 18:27:27.727213 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:27.727175 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58bac243-2634-4106-a98f-66718966a98a-kube-api-access-bchl2" (OuterVolumeSpecName: "kube-api-access-bchl2") pod "58bac243-2634-4106-a98f-66718966a98a" (UID: "58bac243-2634-4106-a98f-66718966a98a"). InnerVolumeSpecName "kube-api-access-bchl2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:27:27.826476 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:27.826442 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bchl2\" (UniqueName: \"kubernetes.io/projected/58bac243-2634-4106-a98f-66718966a98a-kube-api-access-bchl2\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:27:28.445455 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:28.445411 2571 generic.go:356] "Generic (PLEG): container finished" podID="58bac243-2634-4106-a98f-66718966a98a" containerID="fbf35d4f4264cb143f6c01cef44772851400a80c59b6eeba9ee9fa775a899b2b" exitCode=0 Jul 10 18:27:28.445964 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:28.445455 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ptpd9" Jul 10 18:27:28.445964 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:28.445502 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-ptpd9" event={"ID":"58bac243-2634-4106-a98f-66718966a98a","Type":"ContainerDied","Data":"fbf35d4f4264cb143f6c01cef44772851400a80c59b6eeba9ee9fa775a899b2b"} Jul 10 18:27:28.445964 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:28.445673 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-ptpd9" event={"ID":"58bac243-2634-4106-a98f-66718966a98a","Type":"ContainerDied","Data":"40b0b7c8f29ea89757c362cbcccfa1bf83d62dba4304740e1f4cbabf0365719b"} Jul 10 18:27:28.445964 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:28.445701 2571 scope.go:117] "RemoveContainer" containerID="fbf35d4f4264cb143f6c01cef44772851400a80c59b6eeba9ee9fa775a899b2b" Jul 10 18:27:28.447969 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:28.447937 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54c6b9cfb7-jvsxq" event={"ID":"de834122-1105-4eb3-8693-4564ba44d3cd","Type":"ContainerStarted","Data":"842f909282ec0533c0d7cb3d48d17a329ab432c350089a433634d92e5f6535c4"} Jul 10 18:27:28.448106 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:28.448001 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-649dd84959-lslz5" Jul 10 18:27:28.456483 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:28.456459 2571 scope.go:117] "RemoveContainer" containerID="fbf35d4f4264cb143f6c01cef44772851400a80c59b6eeba9ee9fa775a899b2b" Jul 10 18:27:28.456825 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:27:28.456802 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbf35d4f4264cb143f6c01cef44772851400a80c59b6eeba9ee9fa775a899b2b\": container with ID starting with fbf35d4f4264cb143f6c01cef44772851400a80c59b6eeba9ee9fa775a899b2b not found: ID does not exist" containerID="fbf35d4f4264cb143f6c01cef44772851400a80c59b6eeba9ee9fa775a899b2b" Jul 10 18:27:28.456908 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:28.456833 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbf35d4f4264cb143f6c01cef44772851400a80c59b6eeba9ee9fa775a899b2b"} err="failed to get container status \"fbf35d4f4264cb143f6c01cef44772851400a80c59b6eeba9ee9fa775a899b2b\": rpc error: code = NotFound desc = could not find container \"fbf35d4f4264cb143f6c01cef44772851400a80c59b6eeba9ee9fa775a899b2b\": container with ID starting with fbf35d4f4264cb143f6c01cef44772851400a80c59b6eeba9ee9fa775a899b2b not found: ID does not exist" Jul 10 18:27:28.469638 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:28.469568 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-54c6b9cfb7-jvsxq" podStartSLOduration=1.485610265 podStartE2EDuration="2.469551195s" totalImagesPullingTime="983.94093ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:27:28.46857666 +0000 UTC m=+605.367845763" watchObservedRunningTime="2026-07-10 18:27:28.469551195 +0000 UTC m=+605.368820331" Jul 10 18:27:28.494537 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:28.494496 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-649dd84959-lslz5"] Jul 10 18:27:28.500284 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:28.500248 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4hvqc"] Jul 10 18:27:28.500567 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:28.500535 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-4hvqc" podUID="7ca1438a-25eb-4c11-a17d-655b18c4370b" containerName="authorino" containerID="cri-o://b95e976f9e762a8e1682fd7d4f4950745bafdb4aeb02353304724515d3e669b0" gracePeriod=30 Jul 10 18:27:28.503011 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:28.502990 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-649dd84959-lslz5"] Jul 10 18:27:28.518453 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:28.518370 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ptpd9"] Jul 10 18:27:28.523706 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:28.523674 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ptpd9"] Jul 10 18:27:28.767357 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:28.767328 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4hvqc" Jul 10 18:27:28.833597 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:28.833496 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tksj\" (UniqueName: \"kubernetes.io/projected/7ca1438a-25eb-4c11-a17d-655b18c4370b-kube-api-access-2tksj\") pod \"7ca1438a-25eb-4c11-a17d-655b18c4370b\" (UID: \"7ca1438a-25eb-4c11-a17d-655b18c4370b\") " Jul 10 18:27:28.835629 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:28.835592 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ca1438a-25eb-4c11-a17d-655b18c4370b-kube-api-access-2tksj" (OuterVolumeSpecName: "kube-api-access-2tksj") pod "7ca1438a-25eb-4c11-a17d-655b18c4370b" (UID: "7ca1438a-25eb-4c11-a17d-655b18c4370b"). InnerVolumeSpecName "kube-api-access-2tksj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:27:28.934051 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:28.934012 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tksj\" (UniqueName: \"kubernetes.io/projected/7ca1438a-25eb-4c11-a17d-655b18c4370b-kube-api-access-2tksj\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:27:29.453706 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:29.453672 2571 generic.go:356] "Generic (PLEG): container finished" podID="7ca1438a-25eb-4c11-a17d-655b18c4370b" containerID="b95e976f9e762a8e1682fd7d4f4950745bafdb4aeb02353304724515d3e669b0" exitCode=0 Jul 10 18:27:29.454252 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:29.453721 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4hvqc" Jul 10 18:27:29.454252 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:29.453753 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4hvqc" event={"ID":"7ca1438a-25eb-4c11-a17d-655b18c4370b","Type":"ContainerDied","Data":"b95e976f9e762a8e1682fd7d4f4950745bafdb4aeb02353304724515d3e669b0"} Jul 10 18:27:29.454252 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:29.453791 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4hvqc" event={"ID":"7ca1438a-25eb-4c11-a17d-655b18c4370b","Type":"ContainerDied","Data":"dabf80cda296f703c7db5219c5f7ebdea32222f6ec6c49d1e1bda987a1f42386"} Jul 10 18:27:29.454252 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:29.453809 2571 scope.go:117] "RemoveContainer" containerID="b95e976f9e762a8e1682fd7d4f4950745bafdb4aeb02353304724515d3e669b0" Jul 10 18:27:29.462426 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:29.462407 2571 scope.go:117] "RemoveContainer" containerID="b95e976f9e762a8e1682fd7d4f4950745bafdb4aeb02353304724515d3e669b0" Jul 10 18:27:29.462721 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:27:29.462703 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b95e976f9e762a8e1682fd7d4f4950745bafdb4aeb02353304724515d3e669b0\": container with ID starting with b95e976f9e762a8e1682fd7d4f4950745bafdb4aeb02353304724515d3e669b0 not found: ID does not exist" containerID="b95e976f9e762a8e1682fd7d4f4950745bafdb4aeb02353304724515d3e669b0" Jul 10 18:27:29.462784 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:29.462729 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b95e976f9e762a8e1682fd7d4f4950745bafdb4aeb02353304724515d3e669b0"} err="failed to get container status \"b95e976f9e762a8e1682fd7d4f4950745bafdb4aeb02353304724515d3e669b0\": rpc error: code = NotFound desc = could not find container \"b95e976f9e762a8e1682fd7d4f4950745bafdb4aeb02353304724515d3e669b0\": container with ID starting with b95e976f9e762a8e1682fd7d4f4950745bafdb4aeb02353304724515d3e669b0 not found: ID does not exist" Jul 10 18:27:29.474379 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:29.474357 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4hvqc"] Jul 10 18:27:29.478031 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:29.478010 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4hvqc"] Jul 10 18:27:29.683887 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:29.683854 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58bac243-2634-4106-a98f-66718966a98a" path="/var/lib/kubelet/pods/58bac243-2634-4106-a98f-66718966a98a/volumes" Jul 10 18:27:29.684172 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:29.684159 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ca1438a-25eb-4c11-a17d-655b18c4370b" path="/var/lib/kubelet/pods/7ca1438a-25eb-4c11-a17d-655b18c4370b/volumes" Jul 10 18:27:29.684428 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:27:29.684417 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8f30189-86c9-4feb-bc82-6410f891f5ff" path="/var/lib/kubelet/pods/d8f30189-86c9-4feb-bc82-6410f891f5ff/volumes" Jul 10 18:28:15.683364 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:15.683329 2571 generic.go:356] "Generic (PLEG): container finished" podID="31a095f9-f006-476e-b42b-d7682d0f463b" containerID="6d9cfec587c4fa52a230afc078832fab909d2fd3c576f93eba84179c5e29d56b" exitCode=0 Jul 10 18:28:15.683807 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:15.683399 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" event={"ID":"31a095f9-f006-476e-b42b-d7682d0f463b","Type":"ContainerDied","Data":"6d9cfec587c4fa52a230afc078832fab909d2fd3c576f93eba84179c5e29d56b"} Jul 10 18:28:15.683910 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:15.683894 2571 scope.go:117] "RemoveContainer" containerID="6d9cfec587c4fa52a230afc078832fab909d2fd3c576f93eba84179c5e29d56b" Jul 10 18:28:15.684849 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:15.684822 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:28:15.684959 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:15.684886 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:28:15.684959 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:15.684917 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:28:16.690427 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:16.690393 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" event={"ID":"31a095f9-f006-476e-b42b-d7682d0f463b","Type":"ContainerStarted","Data":"641b1cfd84ba04fc4d4954265dfc71224c8f35fc634589cfd99e4cbad2ec34e0"} Jul 10 18:28:17.335631 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:17.335577 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:28:17.336828 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:17.336795 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-msnmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 10 18:28:17.336952 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:17.336855 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" podUID="31a095f9-f006-476e-b42b-d7682d0f463b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 10 18:28:18.336701 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:18.336650 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-msnmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 10 18:28:18.337142 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:18.336727 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" podUID="31a095f9-f006-476e-b42b-d7682d0f463b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 10 18:28:19.336596 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:19.336544 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-msnmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 10 18:28:19.336831 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:19.336641 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" podUID="31a095f9-f006-476e-b42b-d7682d0f463b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 10 18:28:20.336482 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:20.336443 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-msnmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 10 18:28:20.336727 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:20.336512 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" podUID="31a095f9-f006-476e-b42b-d7682d0f463b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 10 18:28:21.340415 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:21.340387 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:28:21.340873 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:21.340728 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:28:21.341577 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:21.341559 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-msnmz" Jul 10 18:28:34.874311 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:34.874277 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5"] Jul 10 18:28:34.874822 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:34.874640 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ca1438a-25eb-4c11-a17d-655b18c4370b" containerName="authorino" Jul 10 18:28:34.874822 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:34.874653 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ca1438a-25eb-4c11-a17d-655b18c4370b" containerName="authorino" Jul 10 18:28:34.874822 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:34.874660 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58bac243-2634-4106-a98f-66718966a98a" containerName="authorino" Jul 10 18:28:34.874822 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:34.874665 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="58bac243-2634-4106-a98f-66718966a98a" containerName="authorino" Jul 10 18:28:34.874822 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:34.874730 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ca1438a-25eb-4c11-a17d-655b18c4370b" containerName="authorino" Jul 10 18:28:34.874822 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:34.874744 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="58bac243-2634-4106-a98f-66718966a98a" containerName="authorino" Jul 10 18:28:34.879167 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:34.879142 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:34.883117 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:34.883091 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-m2g74\"" Jul 10 18:28:34.883279 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:34.883099 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 10 18:28:34.883279 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:34.883145 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 10 18:28:34.883279 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:34.883145 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 10 18:28:34.890431 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:34.890411 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5"] Jul 10 18:28:34.920371 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:34.920334 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcb9bf03-4584-445d-a810-028464c7b9fd-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wxfq5\" (UID: \"bcb9bf03-4584-445d-a810-028464c7b9fd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:34.920371 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:34.920377 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcb9bf03-4584-445d-a810-028464c7b9fd-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wxfq5\" (UID: \"bcb9bf03-4584-445d-a810-028464c7b9fd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:34.920648 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:34.920395 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcb9bf03-4584-445d-a810-028464c7b9fd-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wxfq5\" (UID: \"bcb9bf03-4584-445d-a810-028464c7b9fd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:34.920648 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:34.920412 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcb9bf03-4584-445d-a810-028464c7b9fd-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wxfq5\" (UID: \"bcb9bf03-4584-445d-a810-028464c7b9fd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:34.920648 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:34.920488 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vckrd\" (UniqueName: \"kubernetes.io/projected/bcb9bf03-4584-445d-a810-028464c7b9fd-kube-api-access-vckrd\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wxfq5\" (UID: \"bcb9bf03-4584-445d-a810-028464c7b9fd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:34.920648 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:34.920539 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcb9bf03-4584-445d-a810-028464c7b9fd-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wxfq5\" (UID: \"bcb9bf03-4584-445d-a810-028464c7b9fd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:35.021724 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:35.021669 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcb9bf03-4584-445d-a810-028464c7b9fd-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wxfq5\" (UID: \"bcb9bf03-4584-445d-a810-028464c7b9fd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:35.021724 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:35.021728 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcb9bf03-4584-445d-a810-028464c7b9fd-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wxfq5\" (UID: \"bcb9bf03-4584-445d-a810-028464c7b9fd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:35.021982 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:35.021749 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcb9bf03-4584-445d-a810-028464c7b9fd-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wxfq5\" (UID: \"bcb9bf03-4584-445d-a810-028464c7b9fd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:35.021982 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:35.021765 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcb9bf03-4584-445d-a810-028464c7b9fd-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wxfq5\" (UID: \"bcb9bf03-4584-445d-a810-028464c7b9fd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:35.021982 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:35.021784 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vckrd\" (UniqueName: \"kubernetes.io/projected/bcb9bf03-4584-445d-a810-028464c7b9fd-kube-api-access-vckrd\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wxfq5\" (UID: \"bcb9bf03-4584-445d-a810-028464c7b9fd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:35.021982 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:35.021808 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcb9bf03-4584-445d-a810-028464c7b9fd-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wxfq5\" (UID: \"bcb9bf03-4584-445d-a810-028464c7b9fd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:35.022183 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:35.022138 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcb9bf03-4584-445d-a810-028464c7b9fd-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wxfq5\" (UID: \"bcb9bf03-4584-445d-a810-028464c7b9fd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:35.022183 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:35.022154 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcb9bf03-4584-445d-a810-028464c7b9fd-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wxfq5\" (UID: \"bcb9bf03-4584-445d-a810-028464c7b9fd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:35.022282 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:35.022228 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcb9bf03-4584-445d-a810-028464c7b9fd-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wxfq5\" (UID: \"bcb9bf03-4584-445d-a810-028464c7b9fd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:35.024514 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:35.024479 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcb9bf03-4584-445d-a810-028464c7b9fd-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wxfq5\" (UID: \"bcb9bf03-4584-445d-a810-028464c7b9fd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:35.024744 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:35.024726 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcb9bf03-4584-445d-a810-028464c7b9fd-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wxfq5\" (UID: \"bcb9bf03-4584-445d-a810-028464c7b9fd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:35.030215 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:35.030165 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vckrd\" (UniqueName: \"kubernetes.io/projected/bcb9bf03-4584-445d-a810-028464c7b9fd-kube-api-access-vckrd\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wxfq5\" (UID: \"bcb9bf03-4584-445d-a810-028464c7b9fd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:35.191440 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:35.191347 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:35.330669 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:35.330643 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5"] Jul 10 18:28:35.333259 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:28:35.333228 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcb9bf03_4584_445d_a810_028464c7b9fd.slice/crio-7fd606b47d8e404dec1f6fce52ceeff570b131663a8b14b8da4927b69bc7611b WatchSource:0}: Error finding container 7fd606b47d8e404dec1f6fce52ceeff570b131663a8b14b8da4927b69bc7611b: Status 404 returned error can't find the container with id 7fd606b47d8e404dec1f6fce52ceeff570b131663a8b14b8da4927b69bc7611b Jul 10 18:28:35.334735 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:35.334719 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:28:35.791655 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:35.791589 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" event={"ID":"bcb9bf03-4584-445d-a810-028464c7b9fd","Type":"ContainerStarted","Data":"7fd606b47d8e404dec1f6fce52ceeff570b131663a8b14b8da4927b69bc7611b"} Jul 10 18:28:36.798806 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:36.798768 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" event={"ID":"bcb9bf03-4584-445d-a810-028464c7b9fd","Type":"ContainerStarted","Data":"16e4c8b788f0a6da023f7d840b69d42152fb081e11a9a6b91db2da6c785d3079"} Jul 10 18:28:36.843320 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:36.843257 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" podStartSLOduration=1.754846498 podStartE2EDuration="2.843239012s" totalImagesPullingTime="1.088392514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:28:36.841758633 +0000 UTC m=+673.741027761" watchObservedRunningTime="2026-07-10 18:28:36.843239012 +0000 UTC m=+673.742508120" Jul 10 18:28:45.192047 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:45.192011 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:45.192470 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:45.192073 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:45.204762 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:45.204737 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:28:45.854061 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:28:45.854036 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wxfq5" Jul 10 18:29:08.389501 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.389462 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs"] Jul 10 18:29:08.395337 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.395311 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:08.400236 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.400210 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 10 18:29:08.407465 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.407435 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs"] Jul 10 18:29:08.434481 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.434443 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bqbd\" (UniqueName: \"kubernetes.io/projected/6fa2adcd-bba5-4c01-91b3-7ecef99b5647-kube-api-access-5bqbd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhxbs\" (UID: \"6fa2adcd-bba5-4c01-91b3-7ecef99b5647\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:08.434686 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.434494 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fa2adcd-bba5-4c01-91b3-7ecef99b5647-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhxbs\" (UID: \"6fa2adcd-bba5-4c01-91b3-7ecef99b5647\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:08.434686 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.434537 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fa2adcd-bba5-4c01-91b3-7ecef99b5647-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhxbs\" (UID: \"6fa2adcd-bba5-4c01-91b3-7ecef99b5647\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:08.434686 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.434568 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fa2adcd-bba5-4c01-91b3-7ecef99b5647-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhxbs\" (UID: \"6fa2adcd-bba5-4c01-91b3-7ecef99b5647\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:08.434686 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.434656 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fa2adcd-bba5-4c01-91b3-7ecef99b5647-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhxbs\" (UID: \"6fa2adcd-bba5-4c01-91b3-7ecef99b5647\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:08.434825 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.434687 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fa2adcd-bba5-4c01-91b3-7ecef99b5647-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhxbs\" (UID: \"6fa2adcd-bba5-4c01-91b3-7ecef99b5647\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:08.535479 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.535437 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fa2adcd-bba5-4c01-91b3-7ecef99b5647-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhxbs\" (UID: \"6fa2adcd-bba5-4c01-91b3-7ecef99b5647\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:08.535479 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.535486 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fa2adcd-bba5-4c01-91b3-7ecef99b5647-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhxbs\" (UID: \"6fa2adcd-bba5-4c01-91b3-7ecef99b5647\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:08.535777 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.535518 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bqbd\" (UniqueName: \"kubernetes.io/projected/6fa2adcd-bba5-4c01-91b3-7ecef99b5647-kube-api-access-5bqbd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhxbs\" (UID: \"6fa2adcd-bba5-4c01-91b3-7ecef99b5647\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:08.535777 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.535560 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fa2adcd-bba5-4c01-91b3-7ecef99b5647-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhxbs\" (UID: \"6fa2adcd-bba5-4c01-91b3-7ecef99b5647\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:08.535777 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.535583 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fa2adcd-bba5-4c01-91b3-7ecef99b5647-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhxbs\" (UID: \"6fa2adcd-bba5-4c01-91b3-7ecef99b5647\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:08.535777 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.535634 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fa2adcd-bba5-4c01-91b3-7ecef99b5647-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhxbs\" (UID: \"6fa2adcd-bba5-4c01-91b3-7ecef99b5647\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:08.536080 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.536055 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fa2adcd-bba5-4c01-91b3-7ecef99b5647-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhxbs\" (UID: \"6fa2adcd-bba5-4c01-91b3-7ecef99b5647\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:08.536159 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.536126 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fa2adcd-bba5-4c01-91b3-7ecef99b5647-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhxbs\" (UID: \"6fa2adcd-bba5-4c01-91b3-7ecef99b5647\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:08.536194 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.536125 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fa2adcd-bba5-4c01-91b3-7ecef99b5647-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhxbs\" (UID: \"6fa2adcd-bba5-4c01-91b3-7ecef99b5647\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:08.537920 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.537897 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fa2adcd-bba5-4c01-91b3-7ecef99b5647-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhxbs\" (UID: \"6fa2adcd-bba5-4c01-91b3-7ecef99b5647\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:08.538075 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.538058 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fa2adcd-bba5-4c01-91b3-7ecef99b5647-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhxbs\" (UID: \"6fa2adcd-bba5-4c01-91b3-7ecef99b5647\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:08.544903 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.544877 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bqbd\" (UniqueName: \"kubernetes.io/projected/6fa2adcd-bba5-4c01-91b3-7ecef99b5647-kube-api-access-5bqbd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jhxbs\" (UID: \"6fa2adcd-bba5-4c01-91b3-7ecef99b5647\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:08.706718 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.706604 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:08.848446 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.848413 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs"] Jul 10 18:29:08.851521 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:29:08.851494 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fa2adcd_bba5_4c01_91b3_7ecef99b5647.slice/crio-0e36af4715fe11b15ff6f79e07c2cbc8105d524239c811e63b3002b57dc8b7cf WatchSource:0}: Error finding container 0e36af4715fe11b15ff6f79e07c2cbc8105d524239c811e63b3002b57dc8b7cf: Status 404 returned error can't find the container with id 0e36af4715fe11b15ff6f79e07c2cbc8105d524239c811e63b3002b57dc8b7cf Jul 10 18:29:08.958975 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:08.958881 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" event={"ID":"6fa2adcd-bba5-4c01-91b3-7ecef99b5647","Type":"ContainerStarted","Data":"0e36af4715fe11b15ff6f79e07c2cbc8105d524239c811e63b3002b57dc8b7cf"} Jul 10 18:29:09.965853 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:09.965816 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" event={"ID":"6fa2adcd-bba5-4c01-91b3-7ecef99b5647","Type":"ContainerStarted","Data":"afc8f1e38fe258da093db757b38f93a9cc3c83aa2167a287464c492810f99bf5"} Jul 10 18:29:10.004680 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:10.004608 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" podStartSLOduration=1.770153477 podStartE2EDuration="2.004587792s" totalImagesPullingTime="234.434315ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:29:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:29:09.990914018 +0000 UTC m=+706.890183126" watchObservedRunningTime="2026-07-10 18:29:10.004587792 +0000 UTC m=+706.903856949" Jul 10 18:29:18.707376 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:18.707262 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:18.707376 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:18.707356 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:18.720156 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:18.720122 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:29:19.030240 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:29:19.030213 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jhxbs" Jul 10 18:30:43.382338 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:43.382290 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7dd8f4d8b-bzwj7"] Jul 10 18:30:43.386130 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:43.386103 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7dd8f4d8b-bzwj7" Jul 10 18:30:43.394197 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:43.394168 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7dd8f4d8b-bzwj7"] Jul 10 18:30:43.453638 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:43.453584 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrj42\" (UniqueName: \"kubernetes.io/projected/246fbc8f-4f72-43b9-b237-3836e6abf287-kube-api-access-hrj42\") pod \"authorino-7dd8f4d8b-bzwj7\" (UID: \"246fbc8f-4f72-43b9-b237-3836e6abf287\") " pod="kuadrant-system/authorino-7dd8f4d8b-bzwj7" Jul 10 18:30:43.453810 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:43.453688 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/246fbc8f-4f72-43b9-b237-3836e6abf287-tls-cert\") pod \"authorino-7dd8f4d8b-bzwj7\" (UID: \"246fbc8f-4f72-43b9-b237-3836e6abf287\") " pod="kuadrant-system/authorino-7dd8f4d8b-bzwj7" Jul 10 18:30:43.554968 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:43.554927 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrj42\" (UniqueName: \"kubernetes.io/projected/246fbc8f-4f72-43b9-b237-3836e6abf287-kube-api-access-hrj42\") pod \"authorino-7dd8f4d8b-bzwj7\" (UID: \"246fbc8f-4f72-43b9-b237-3836e6abf287\") " pod="kuadrant-system/authorino-7dd8f4d8b-bzwj7" Jul 10 18:30:43.555137 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:43.554980 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/246fbc8f-4f72-43b9-b237-3836e6abf287-tls-cert\") pod \"authorino-7dd8f4d8b-bzwj7\" (UID: \"246fbc8f-4f72-43b9-b237-3836e6abf287\") " pod="kuadrant-system/authorino-7dd8f4d8b-bzwj7" Jul 10 18:30:43.557355 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:43.557336 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/246fbc8f-4f72-43b9-b237-3836e6abf287-tls-cert\") pod \"authorino-7dd8f4d8b-bzwj7\" (UID: \"246fbc8f-4f72-43b9-b237-3836e6abf287\") " pod="kuadrant-system/authorino-7dd8f4d8b-bzwj7" Jul 10 18:30:43.567080 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:43.567053 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrj42\" (UniqueName: \"kubernetes.io/projected/246fbc8f-4f72-43b9-b237-3836e6abf287-kube-api-access-hrj42\") pod \"authorino-7dd8f4d8b-bzwj7\" (UID: \"246fbc8f-4f72-43b9-b237-3836e6abf287\") " pod="kuadrant-system/authorino-7dd8f4d8b-bzwj7" Jul 10 18:30:43.696413 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:43.696325 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7dd8f4d8b-bzwj7" Jul 10 18:30:43.848483 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:43.848459 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7dd8f4d8b-bzwj7"] Jul 10 18:30:43.850754 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:30:43.850719 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod246fbc8f_4f72_43b9_b237_3836e6abf287.slice/crio-b7817e9c7d3c331f71ef99a32d105fab91255d87171c6d20f4c41fb716667bca WatchSource:0}: Error finding container b7817e9c7d3c331f71ef99a32d105fab91255d87171c6d20f4c41fb716667bca: Status 404 returned error can't find the container with id b7817e9c7d3c331f71ef99a32d105fab91255d87171c6d20f4c41fb716667bca Jul 10 18:30:44.443022 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:44.442985 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7dd8f4d8b-bzwj7" event={"ID":"246fbc8f-4f72-43b9-b237-3836e6abf287","Type":"ContainerStarted","Data":"b7817e9c7d3c331f71ef99a32d105fab91255d87171c6d20f4c41fb716667bca"} Jul 10 18:30:45.452415 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:45.452296 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7dd8f4d8b-bzwj7" event={"ID":"246fbc8f-4f72-43b9-b237-3836e6abf287","Type":"ContainerStarted","Data":"d4e90c2c06fe9a18424e5b89cd75b018ed9b40ee994139d115a6ebeab8723fc8"} Jul 10 18:30:45.472461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:45.472401 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7dd8f4d8b-bzwj7" podStartSLOduration=1.322098133 podStartE2EDuration="2.472380828s" totalImagesPullingTime="1.150282695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:30:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:30:45.470160447 +0000 UTC m=+802.369429556" watchObservedRunningTime="2026-07-10 18:30:45.472380828 +0000 UTC m=+802.371649936" Jul 10 18:30:45.507500 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:45.507460 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54c6b9cfb7-jvsxq"] Jul 10 18:30:45.508131 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:45.508060 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-54c6b9cfb7-jvsxq" podUID="de834122-1105-4eb3-8693-4564ba44d3cd" containerName="authorino" containerID="cri-o://842f909282ec0533c0d7cb3d48d17a329ab432c350089a433634d92e5f6535c4" gracePeriod=30 Jul 10 18:30:45.794323 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:45.794292 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54c6b9cfb7-jvsxq" Jul 10 18:30:45.878821 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:45.878788 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/de834122-1105-4eb3-8693-4564ba44d3cd-tls-cert\") pod \"de834122-1105-4eb3-8693-4564ba44d3cd\" (UID: \"de834122-1105-4eb3-8693-4564ba44d3cd\") " Jul 10 18:30:45.879000 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:45.878832 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzxz9\" (UniqueName: \"kubernetes.io/projected/de834122-1105-4eb3-8693-4564ba44d3cd-kube-api-access-gzxz9\") pod \"de834122-1105-4eb3-8693-4564ba44d3cd\" (UID: \"de834122-1105-4eb3-8693-4564ba44d3cd\") " Jul 10 18:30:45.881294 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:45.881261 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de834122-1105-4eb3-8693-4564ba44d3cd-kube-api-access-gzxz9" (OuterVolumeSpecName: "kube-api-access-gzxz9") pod "de834122-1105-4eb3-8693-4564ba44d3cd" (UID: "de834122-1105-4eb3-8693-4564ba44d3cd"). InnerVolumeSpecName "kube-api-access-gzxz9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:30:45.889964 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:45.889928 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de834122-1105-4eb3-8693-4564ba44d3cd-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "de834122-1105-4eb3-8693-4564ba44d3cd" (UID: "de834122-1105-4eb3-8693-4564ba44d3cd"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:30:45.980367 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:45.980283 2571 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/de834122-1105-4eb3-8693-4564ba44d3cd-tls-cert\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:30:45.980367 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:45.980314 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzxz9\" (UniqueName: \"kubernetes.io/projected/de834122-1105-4eb3-8693-4564ba44d3cd-kube-api-access-gzxz9\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:30:46.459915 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:46.459878 2571 generic.go:356] "Generic (PLEG): container finished" podID="de834122-1105-4eb3-8693-4564ba44d3cd" containerID="842f909282ec0533c0d7cb3d48d17a329ab432c350089a433634d92e5f6535c4" exitCode=0 Jul 10 18:30:46.459915 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:46.459913 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54c6b9cfb7-jvsxq" Jul 10 18:30:46.460475 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:46.459950 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54c6b9cfb7-jvsxq" event={"ID":"de834122-1105-4eb3-8693-4564ba44d3cd","Type":"ContainerDied","Data":"842f909282ec0533c0d7cb3d48d17a329ab432c350089a433634d92e5f6535c4"} Jul 10 18:30:46.460475 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:46.459997 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54c6b9cfb7-jvsxq" event={"ID":"de834122-1105-4eb3-8693-4564ba44d3cd","Type":"ContainerDied","Data":"065a0cf9904ab771baa1bf1e6c47f239bbacab5bfc2b1241eb1380af8ab25e14"} Jul 10 18:30:46.460475 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:46.460018 2571 scope.go:117] "RemoveContainer" containerID="842f909282ec0533c0d7cb3d48d17a329ab432c350089a433634d92e5f6535c4" Jul 10 18:30:46.470887 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:46.470869 2571 scope.go:117] "RemoveContainer" containerID="842f909282ec0533c0d7cb3d48d17a329ab432c350089a433634d92e5f6535c4" Jul 10 18:30:46.471296 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:30:46.471280 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"842f909282ec0533c0d7cb3d48d17a329ab432c350089a433634d92e5f6535c4\": container with ID starting with 842f909282ec0533c0d7cb3d48d17a329ab432c350089a433634d92e5f6535c4 not found: ID does not exist" containerID="842f909282ec0533c0d7cb3d48d17a329ab432c350089a433634d92e5f6535c4" Jul 10 18:30:46.471347 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:46.471306 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"842f909282ec0533c0d7cb3d48d17a329ab432c350089a433634d92e5f6535c4"} err="failed to get container status \"842f909282ec0533c0d7cb3d48d17a329ab432c350089a433634d92e5f6535c4\": rpc error: code = NotFound desc = could not find container \"842f909282ec0533c0d7cb3d48d17a329ab432c350089a433634d92e5f6535c4\": container with ID starting with 842f909282ec0533c0d7cb3d48d17a329ab432c350089a433634d92e5f6535c4 not found: ID does not exist" Jul 10 18:30:46.489100 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:46.489066 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54c6b9cfb7-jvsxq"] Jul 10 18:30:46.491282 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:46.491259 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-54c6b9cfb7-jvsxq"] Jul 10 18:30:47.683514 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:30:47.683481 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de834122-1105-4eb3-8693-4564ba44d3cd" path="/var/lib/kubelet/pods/de834122-1105-4eb3-8693-4564ba44d3cd/volumes" Jul 10 18:32:23.568149 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:32:23.568121 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r9vz_0cafad48-3998-465d-b4c8-b5d94cac6b3c/ovn-acl-logging/0.log" Jul 10 18:32:23.570012 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:32:23.569973 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r9vz_0cafad48-3998-465d-b4c8-b5d94cac6b3c/ovn-acl-logging/0.log" Jul 10 18:36:54.705227 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:36:54.705137 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f"] Jul 10 18:36:54.706084 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:36:54.705355 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f" podUID="ed995b0f-5598-4cb9-a0f9-5f84d083cb3b" containerName="manager" containerID="cri-o://d3bd6d11a4e5c6d366a93a16f0e706e8faf44192ce7d3378b4a8120756c18ab5" gracePeriod=10 Jul 10 18:36:59.077476 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:36:59.077402 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f" Jul 10 18:36:59.199359 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:36:59.199322 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed995b0f-5598-4cb9-a0f9-5f84d083cb3b-extensions-socket-volume\") pod \"ed995b0f-5598-4cb9-a0f9-5f84d083cb3b\" (UID: \"ed995b0f-5598-4cb9-a0f9-5f84d083cb3b\") " Jul 10 18:36:59.199530 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:36:59.199377 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vc92c\" (UniqueName: \"kubernetes.io/projected/ed995b0f-5598-4cb9-a0f9-5f84d083cb3b-kube-api-access-vc92c\") pod \"ed995b0f-5598-4cb9-a0f9-5f84d083cb3b\" (UID: \"ed995b0f-5598-4cb9-a0f9-5f84d083cb3b\") " Jul 10 18:36:59.199746 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:36:59.199721 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed995b0f-5598-4cb9-a0f9-5f84d083cb3b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ed995b0f-5598-4cb9-a0f9-5f84d083cb3b" (UID: "ed995b0f-5598-4cb9-a0f9-5f84d083cb3b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:36:59.201514 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:36:59.201485 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed995b0f-5598-4cb9-a0f9-5f84d083cb3b-kube-api-access-vc92c" (OuterVolumeSpecName: "kube-api-access-vc92c") pod "ed995b0f-5598-4cb9-a0f9-5f84d083cb3b" (UID: "ed995b0f-5598-4cb9-a0f9-5f84d083cb3b"). InnerVolumeSpecName "kube-api-access-vc92c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:36:59.300984 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:36:59.300939 2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed995b0f-5598-4cb9-a0f9-5f84d083cb3b-extensions-socket-volume\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:36:59.300984 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:36:59.300975 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vc92c\" (UniqueName: \"kubernetes.io/projected/ed995b0f-5598-4cb9-a0f9-5f84d083cb3b-kube-api-access-vc92c\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:36:59.399155 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:36:59.399061 2571 generic.go:356] "Generic (PLEG): container finished" podID="ed995b0f-5598-4cb9-a0f9-5f84d083cb3b" containerID="d3bd6d11a4e5c6d366a93a16f0e706e8faf44192ce7d3378b4a8120756c18ab5" exitCode=0 Jul 10 18:36:59.399155 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:36:59.399118 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f" Jul 10 18:36:59.399155 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:36:59.399144 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f" event={"ID":"ed995b0f-5598-4cb9-a0f9-5f84d083cb3b","Type":"ContainerDied","Data":"d3bd6d11a4e5c6d366a93a16f0e706e8faf44192ce7d3378b4a8120756c18ab5"} Jul 10 18:36:59.399389 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:36:59.399201 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f" event={"ID":"ed995b0f-5598-4cb9-a0f9-5f84d083cb3b","Type":"ContainerDied","Data":"42e35e37be57ec26552d0da9dcbfc4f6d543991d3db8671f6d743a903f5d08b3"} Jul 10 18:36:59.399389 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:36:59.399227 2571 scope.go:117] "RemoveContainer" containerID="d3bd6d11a4e5c6d366a93a16f0e706e8faf44192ce7d3378b4a8120756c18ab5" Jul 10 18:36:59.410184 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:36:59.410165 2571 scope.go:117] "RemoveContainer" containerID="d3bd6d11a4e5c6d366a93a16f0e706e8faf44192ce7d3378b4a8120756c18ab5" Jul 10 18:36:59.410490 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:36:59.410471 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3bd6d11a4e5c6d366a93a16f0e706e8faf44192ce7d3378b4a8120756c18ab5\": container with ID starting with d3bd6d11a4e5c6d366a93a16f0e706e8faf44192ce7d3378b4a8120756c18ab5 not found: ID does not exist" containerID="d3bd6d11a4e5c6d366a93a16f0e706e8faf44192ce7d3378b4a8120756c18ab5" Jul 10 18:36:59.410564 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:36:59.410507 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3bd6d11a4e5c6d366a93a16f0e706e8faf44192ce7d3378b4a8120756c18ab5"} err="failed to get container status \"d3bd6d11a4e5c6d366a93a16f0e706e8faf44192ce7d3378b4a8120756c18ab5\": rpc error: code = NotFound desc = could not find container \"d3bd6d11a4e5c6d366a93a16f0e706e8faf44192ce7d3378b4a8120756c18ab5\": container with ID starting with d3bd6d11a4e5c6d366a93a16f0e706e8faf44192ce7d3378b4a8120756c18ab5 not found: ID does not exist" Jul 10 18:36:59.429085 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:36:59.429056 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f"] Jul 10 18:36:59.436850 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:36:59.436824 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnd7f"] Jul 10 18:36:59.683989 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:36:59.683909 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed995b0f-5598-4cb9-a0f9-5f84d083cb3b" path="/var/lib/kubelet/pods/ed995b0f-5598-4cb9-a0f9-5f84d083cb3b/volumes" Jul 10 18:37:23.600389 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:23.600359 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r9vz_0cafad48-3998-465d-b4c8-b5d94cac6b3c/ovn-acl-logging/0.log" Jul 10 18:37:23.601853 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:23.601834 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r9vz_0cafad48-3998-465d-b4c8-b5d94cac6b3c/ovn-acl-logging/0.log" Jul 10 18:37:55.854744 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:55.854706 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xh2np"] Jul 10 18:37:55.855393 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:55.855218 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed995b0f-5598-4cb9-a0f9-5f84d083cb3b" containerName="manager" Jul 10 18:37:55.855393 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:55.855234 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed995b0f-5598-4cb9-a0f9-5f84d083cb3b" containerName="manager" Jul 10 18:37:55.855393 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:55.855263 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de834122-1105-4eb3-8693-4564ba44d3cd" containerName="authorino" Jul 10 18:37:55.855393 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:55.855271 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="de834122-1105-4eb3-8693-4564ba44d3cd" containerName="authorino" Jul 10 18:37:55.855393 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:55.855368 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="de834122-1105-4eb3-8693-4564ba44d3cd" containerName="authorino" Jul 10 18:37:55.855393 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:55.855380 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed995b0f-5598-4cb9-a0f9-5f84d083cb3b" containerName="manager" Jul 10 18:37:55.858469 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:55.858449 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xh2np" Jul 10 18:37:55.861794 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:55.861773 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-47r9g\"" Jul 10 18:37:55.870991 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:55.870958 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xh2np"] Jul 10 18:37:55.981144 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:55.981109 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/164d1efe-3fe0-4f7d-9fd6-1031e530a9d8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xh2np\" (UID: \"164d1efe-3fe0-4f7d-9fd6-1031e530a9d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xh2np" Jul 10 18:37:55.981347 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:55.981183 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcqs8\" (UniqueName: \"kubernetes.io/projected/164d1efe-3fe0-4f7d-9fd6-1031e530a9d8-kube-api-access-fcqs8\") pod \"kuadrant-operator-controller-manager-86878c769f-xh2np\" (UID: \"164d1efe-3fe0-4f7d-9fd6-1031e530a9d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xh2np" Jul 10 18:37:56.082355 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:56.082305 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/164d1efe-3fe0-4f7d-9fd6-1031e530a9d8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xh2np\" (UID: \"164d1efe-3fe0-4f7d-9fd6-1031e530a9d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xh2np" Jul 10 18:37:56.082526 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:56.082412 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcqs8\" (UniqueName: \"kubernetes.io/projected/164d1efe-3fe0-4f7d-9fd6-1031e530a9d8-kube-api-access-fcqs8\") pod \"kuadrant-operator-controller-manager-86878c769f-xh2np\" (UID: \"164d1efe-3fe0-4f7d-9fd6-1031e530a9d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xh2np" Jul 10 18:37:56.082841 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:56.082817 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/164d1efe-3fe0-4f7d-9fd6-1031e530a9d8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xh2np\" (UID: \"164d1efe-3fe0-4f7d-9fd6-1031e530a9d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xh2np" Jul 10 18:37:56.101580 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:56.101542 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcqs8\" (UniqueName: \"kubernetes.io/projected/164d1efe-3fe0-4f7d-9fd6-1031e530a9d8-kube-api-access-fcqs8\") pod \"kuadrant-operator-controller-manager-86878c769f-xh2np\" (UID: \"164d1efe-3fe0-4f7d-9fd6-1031e530a9d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xh2np" Jul 10 18:37:56.170217 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:56.170121 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xh2np" Jul 10 18:37:56.316466 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:56.316443 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xh2np"] Jul 10 18:37:56.319275 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:37:56.319245 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod164d1efe_3fe0_4f7d_9fd6_1031e530a9d8.slice/crio-57c4a8cbfd79e08d9d2153cd22164d8bd9804a67b9fb6783fcb4d271725b910a WatchSource:0}: Error finding container 57c4a8cbfd79e08d9d2153cd22164d8bd9804a67b9fb6783fcb4d271725b910a: Status 404 returned error can't find the container with id 57c4a8cbfd79e08d9d2153cd22164d8bd9804a67b9fb6783fcb4d271725b910a Jul 10 18:37:56.689173 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:56.689133 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xh2np" event={"ID":"164d1efe-3fe0-4f7d-9fd6-1031e530a9d8","Type":"ContainerStarted","Data":"7ca9021c8d8f67a382ce66ab26b662f748f4a6d5e00c1e8697e237e5a1ea8d47"} Jul 10 18:37:56.689357 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:56.689188 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xh2np" event={"ID":"164d1efe-3fe0-4f7d-9fd6-1031e530a9d8","Type":"ContainerStarted","Data":"57c4a8cbfd79e08d9d2153cd22164d8bd9804a67b9fb6783fcb4d271725b910a"} Jul 10 18:37:56.689357 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:56.689211 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xh2np" Jul 10 18:37:56.711928 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:37:56.711868 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xh2np" podStartSLOduration=1.711850885 podStartE2EDuration="1.711850885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:37:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:37:56.71037706 +0000 UTC m=+1233.609646169" watchObservedRunningTime="2026-07-10 18:37:56.711850885 +0000 UTC m=+1233.611119987" Jul 10 18:38:07.697188 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:38:07.697159 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xh2np" Jul 10 18:42:23.635146 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:42:23.635118 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r9vz_0cafad48-3998-465d-b4c8-b5d94cac6b3c/ovn-acl-logging/0.log" Jul 10 18:42:23.637683 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:42:23.637412 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r9vz_0cafad48-3998-465d-b4c8-b5d94cac6b3c/ovn-acl-logging/0.log" Jul 10 18:44:27.202107 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.202075 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5"] Jul 10 18:44:27.205887 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.205862 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.209414 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.209389 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8ca6169c-openshift-default-dockercfg-n8wn9\"" Jul 10 18:44:27.226027 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.225993 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5"] Jul 10 18:44:27.290175 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.290137 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af34555d-dfb6-4130-93b4-6e969b9e62f6-istiod-ca-cert\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.290175 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.290174 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-podinfo\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.290464 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.290207 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-workload-certs\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.290464 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.290305 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-data\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.290464 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.290348 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-token\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.290464 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.290378 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-credential-socket\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.290464 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.290433 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-workload-socket\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.290693 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.290471 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kdh2\" (UniqueName: \"kubernetes.io/projected/af34555d-dfb6-4130-93b4-6e969b9e62f6-kube-api-access-4kdh2\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.290693 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.290508 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-envoy\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.391697 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.391640 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af34555d-dfb6-4130-93b4-6e969b9e62f6-istiod-ca-cert\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.391697 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.391702 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-podinfo\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.391946 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.391736 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-workload-certs\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.391946 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.391775 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-data\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.391946 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.391791 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-token\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.391946 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.391818 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-credential-socket\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.391946 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.391871 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-workload-socket\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.391946 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.391900 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kdh2\" (UniqueName: \"kubernetes.io/projected/af34555d-dfb6-4130-93b4-6e969b9e62f6-kube-api-access-4kdh2\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.392222 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.391965 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-envoy\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.392222 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.392153 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-workload-certs\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.392354 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.392240 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-data\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.392354 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.392295 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-workload-socket\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.392460 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.392392 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-credential-socket\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.392498 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.392453 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af34555d-dfb6-4130-93b4-6e969b9e62f6-istiod-ca-cert\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.394246 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.394222 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-envoy\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.394404 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.394389 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-podinfo\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.399859 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.399835 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-token\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.399952 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.399934 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kdh2\" (UniqueName: \"kubernetes.io/projected/af34555d-dfb6-4130-93b4-6e969b9e62f6-kube-api-access-4kdh2\") pod \"e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.517066 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.517013 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:27.658256 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.658223 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5"] Jul 10 18:44:27.659649 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:44:27.659601 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf34555d_dfb6_4130_93b4_6e969b9e62f6.slice/crio-4a6ba007398e99fb0aaabc3eef8d56303c49ededdb760cedeb814dad88c0503f WatchSource:0}: Error finding container 4a6ba007398e99fb0aaabc3eef8d56303c49ededdb760cedeb814dad88c0503f: Status 404 returned error can't find the container with id 4a6ba007398e99fb0aaabc3eef8d56303c49ededdb760cedeb814dad88c0503f Jul 10 18:44:27.662000 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.661960 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:44:27.662086 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.662053 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:44:27.662125 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.662095 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:44:27.735846 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.735807 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" event={"ID":"af34555d-dfb6-4130-93b4-6e969b9e62f6","Type":"ContainerStarted","Data":"6952402bed55f0c482e7f2ef1124698a3ab42092f2f4e47cad937037ba1e84bd"} Jul 10 18:44:27.736038 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.735876 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" event={"ID":"af34555d-dfb6-4130-93b4-6e969b9e62f6","Type":"ContainerStarted","Data":"4a6ba007398e99fb0aaabc3eef8d56303c49ededdb760cedeb814dad88c0503f"} Jul 10 18:44:27.759188 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:27.759132 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" podStartSLOduration=0.759113676 podStartE2EDuration="759.113676ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:44:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:44:27.756805319 +0000 UTC m=+1624.656074454" watchObservedRunningTime="2026-07-10 18:44:27.759113676 +0000 UTC m=+1624.658382837" Jul 10 18:44:28.518069 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:28.518035 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:28.524258 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:28.524220 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:28.741573 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:28.741542 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:28.742569 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:28.742547 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:33.795626 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:33.795567 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5"] Jul 10 18:44:33.796184 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:33.795810 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" podUID="af34555d-dfb6-4130-93b4-6e969b9e62f6" containerName="istio-proxy" containerID="cri-o://6952402bed55f0c482e7f2ef1124698a3ab42092f2f4e47cad937037ba1e84bd" gracePeriod=30 Jul 10 18:44:39.808711 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:39.808674 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd"] Jul 10 18:44:39.815195 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:39.815162 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:39.819900 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:39.819871 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8dfa64c8-openshift-default-dockercfg-t7dvv\"" Jul 10 18:44:39.829828 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:39.829799 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd"] Jul 10 18:44:39.914796 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:39.914761 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhrdl\" (UniqueName: \"kubernetes.io/projected/5a859741-03fa-4482-8eb7-a6a93a976ea8-kube-api-access-jhrdl\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:39.915796 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:39.914814 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-envoy\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:39.915796 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:39.914851 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-credential-socket\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:39.915796 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:39.914904 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-podinfo\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:39.915796 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:39.914944 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a859741-03fa-4482-8eb7-a6a93a976ea8-istiod-ca-cert\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:39.915796 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:39.914968 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-token\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:39.915796 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:39.915067 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-data\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:39.915796 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:39.915143 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-workload-socket\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:39.915796 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:39.915196 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-workload-certs\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:40.016534 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.016497 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-podinfo\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:40.016679 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.016555 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a859741-03fa-4482-8eb7-a6a93a976ea8-istiod-ca-cert\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:40.016679 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.016587 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-token\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:40.016679 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.016644 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-data\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:40.016848 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.016696 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-workload-socket\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:40.016848 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.016752 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-workload-certs\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:40.016848 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.016784 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhrdl\" (UniqueName: \"kubernetes.io/projected/5a859741-03fa-4482-8eb7-a6a93a976ea8-kube-api-access-jhrdl\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:40.016848 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.016812 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-envoy\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:40.016848 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.016839 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-credential-socket\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:40.017835 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.017271 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-workload-socket\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:40.017835 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.017439 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a859741-03fa-4482-8eb7-a6a93a976ea8-istiod-ca-cert\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:40.017835 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.017523 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-data\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:40.017835 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.017744 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-workload-certs\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:40.018305 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.018280 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-credential-socket\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:40.019291 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.019271 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-podinfo\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:40.020497 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.020478 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-envoy\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:40.025153 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.025124 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-token\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:40.025275 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.025256 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhrdl\" (UniqueName: \"kubernetes.io/projected/5a859741-03fa-4482-8eb7-a6a93a976ea8-kube-api-access-jhrdl\") pod \"e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:40.129124 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.129091 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:40.169036 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.168987 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:40.277594 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.277563 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd"] Jul 10 18:44:40.279077 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:44:40.279051 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a859741_03fa_4482_8eb7_a6a93a976ea8.slice/crio-1573442b2b0cad38a7206d4ecc1fbc6b8b19f3aa29a146fa4ec1e06052b9f454 WatchSource:0}: Error finding container 1573442b2b0cad38a7206d4ecc1fbc6b8b19f3aa29a146fa4ec1e06052b9f454: Status 404 returned error can't find the container with id 1573442b2b0cad38a7206d4ecc1fbc6b8b19f3aa29a146fa4ec1e06052b9f454 Jul 10 18:44:40.281165 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.281137 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:44:40.281216 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.281200 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:44:40.281250 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.281239 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:44:40.320641 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.320592 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-token\") pod \"af34555d-dfb6-4130-93b4-6e969b9e62f6\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " Jul 10 18:44:40.320792 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.320663 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-workload-socket\") pod \"af34555d-dfb6-4130-93b4-6e969b9e62f6\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " Jul 10 18:44:40.320792 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.320700 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-data\") pod \"af34555d-dfb6-4130-93b4-6e969b9e62f6\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " Jul 10 18:44:40.320792 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.320757 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-envoy\") pod \"af34555d-dfb6-4130-93b4-6e969b9e62f6\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " Jul 10 18:44:40.320792 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.320786 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-workload-certs\") pod \"af34555d-dfb6-4130-93b4-6e969b9e62f6\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " Jul 10 18:44:40.320987 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.320866 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-credential-socket\") pod \"af34555d-dfb6-4130-93b4-6e969b9e62f6\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " Jul 10 18:44:40.320987 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.320898 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af34555d-dfb6-4130-93b4-6e969b9e62f6-istiod-ca-cert\") pod \"af34555d-dfb6-4130-93b4-6e969b9e62f6\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " Jul 10 18:44:40.320987 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.320926 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-podinfo\") pod \"af34555d-dfb6-4130-93b4-6e969b9e62f6\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " Jul 10 18:44:40.320987 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.320954 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "af34555d-dfb6-4130-93b4-6e969b9e62f6" (UID: "af34555d-dfb6-4130-93b4-6e969b9e62f6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:44:40.320987 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.320964 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kdh2\" (UniqueName: \"kubernetes.io/projected/af34555d-dfb6-4130-93b4-6e969b9e62f6-kube-api-access-4kdh2\") pod \"af34555d-dfb6-4130-93b4-6e969b9e62f6\" (UID: \"af34555d-dfb6-4130-93b4-6e969b9e62f6\") " Jul 10 18:44:40.321186 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.321151 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "af34555d-dfb6-4130-93b4-6e969b9e62f6" (UID: "af34555d-dfb6-4130-93b4-6e969b9e62f6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:44:40.321373 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.321326 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "af34555d-dfb6-4130-93b4-6e969b9e62f6" (UID: "af34555d-dfb6-4130-93b4-6e969b9e62f6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:44:40.321462 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.321353 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-workload-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:44:40.321462 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.321398 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-credential-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:44:40.321571 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.321470 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af34555d-dfb6-4130-93b4-6e969b9e62f6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "af34555d-dfb6-4130-93b4-6e969b9e62f6" (UID: "af34555d-dfb6-4130-93b4-6e969b9e62f6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:44:40.321571 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.321505 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-data" (OuterVolumeSpecName: "istio-data") pod "af34555d-dfb6-4130-93b4-6e969b9e62f6" (UID: "af34555d-dfb6-4130-93b4-6e969b9e62f6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:44:40.323304 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.323248 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "af34555d-dfb6-4130-93b4-6e969b9e62f6" (UID: "af34555d-dfb6-4130-93b4-6e969b9e62f6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:44:40.323406 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.323310 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af34555d-dfb6-4130-93b4-6e969b9e62f6-kube-api-access-4kdh2" (OuterVolumeSpecName: "kube-api-access-4kdh2") pod "af34555d-dfb6-4130-93b4-6e969b9e62f6" (UID: "af34555d-dfb6-4130-93b4-6e969b9e62f6"). InnerVolumeSpecName "kube-api-access-4kdh2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:44:40.323573 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.323553 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "af34555d-dfb6-4130-93b4-6e969b9e62f6" (UID: "af34555d-dfb6-4130-93b4-6e969b9e62f6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 18:44:40.323709 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.323691 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-token" (OuterVolumeSpecName: "istio-token") pod "af34555d-dfb6-4130-93b4-6e969b9e62f6" (UID: "af34555d-dfb6-4130-93b4-6e969b9e62f6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:44:40.422386 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.422357 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af34555d-dfb6-4130-93b4-6e969b9e62f6-istiod-ca-cert\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:44:40.422386 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.422382 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-podinfo\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:44:40.422386 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.422391 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kdh2\" (UniqueName: \"kubernetes.io/projected/af34555d-dfb6-4130-93b4-6e969b9e62f6-kube-api-access-4kdh2\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:44:40.422704 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.422402 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-token\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:44:40.422704 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.422413 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-data\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:44:40.422704 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.422421 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-istio-envoy\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:44:40.422704 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.422429 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af34555d-dfb6-4130-93b4-6e969b9e62f6-workload-certs\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:44:40.815294 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.815259 2571 generic.go:356] "Generic (PLEG): container finished" podID="af34555d-dfb6-4130-93b4-6e969b9e62f6" containerID="6952402bed55f0c482e7f2ef1124698a3ab42092f2f4e47cad937037ba1e84bd" exitCode=0 Jul 10 18:44:40.815809 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.815334 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" Jul 10 18:44:40.815809 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.815420 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" event={"ID":"af34555d-dfb6-4130-93b4-6e969b9e62f6","Type":"ContainerDied","Data":"6952402bed55f0c482e7f2ef1124698a3ab42092f2f4e47cad937037ba1e84bd"} Jul 10 18:44:40.815809 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.815498 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5" event={"ID":"af34555d-dfb6-4130-93b4-6e969b9e62f6","Type":"ContainerDied","Data":"4a6ba007398e99fb0aaabc3eef8d56303c49ededdb760cedeb814dad88c0503f"} Jul 10 18:44:40.815809 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.815528 2571 scope.go:117] "RemoveContainer" containerID="6952402bed55f0c482e7f2ef1124698a3ab42092f2f4e47cad937037ba1e84bd" Jul 10 18:44:40.818292 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.818263 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" event={"ID":"5a859741-03fa-4482-8eb7-a6a93a976ea8","Type":"ContainerStarted","Data":"9cb5d2946acaf2eac6dde31b0429cba5d847d7cad54efb3ee302fdf34fcf38fc"} Jul 10 18:44:40.818481 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.818460 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" event={"ID":"5a859741-03fa-4482-8eb7-a6a93a976ea8","Type":"ContainerStarted","Data":"1573442b2b0cad38a7206d4ecc1fbc6b8b19f3aa29a146fa4ec1e06052b9f454"} Jul 10 18:44:40.828994 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.828970 2571 scope.go:117] "RemoveContainer" containerID="6952402bed55f0c482e7f2ef1124698a3ab42092f2f4e47cad937037ba1e84bd" Jul 10 18:44:40.830779 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:44:40.830750 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6952402bed55f0c482e7f2ef1124698a3ab42092f2f4e47cad937037ba1e84bd\": container with ID starting with 6952402bed55f0c482e7f2ef1124698a3ab42092f2f4e47cad937037ba1e84bd not found: ID does not exist" containerID="6952402bed55f0c482e7f2ef1124698a3ab42092f2f4e47cad937037ba1e84bd" Jul 10 18:44:40.830888 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.830790 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6952402bed55f0c482e7f2ef1124698a3ab42092f2f4e47cad937037ba1e84bd"} err="failed to get container status \"6952402bed55f0c482e7f2ef1124698a3ab42092f2f4e47cad937037ba1e84bd\": rpc error: code = NotFound desc = could not find container \"6952402bed55f0c482e7f2ef1124698a3ab42092f2f4e47cad937037ba1e84bd\": container with ID starting with 6952402bed55f0c482e7f2ef1124698a3ab42092f2f4e47cad937037ba1e84bd not found: ID does not exist" Jul 10 18:44:40.848428 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.848379 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" podStartSLOduration=1.8483632 podStartE2EDuration="1.8483632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:44:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:44:40.846734617 +0000 UTC m=+1637.746003745" watchObservedRunningTime="2026-07-10 18:44:40.8483632 +0000 UTC m=+1637.747632308" Jul 10 18:44:40.866635 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.866582 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5"] Jul 10 18:44:40.871368 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:40.871342 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8ca6169c-openshift-default-666cfbc665-qqhd5"] Jul 10 18:44:41.129851 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:41.129759 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:41.134660 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:41.134632 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:41.683719 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:41.683684 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af34555d-dfb6-4130-93b4-6e969b9e62f6" path="/var/lib/kubelet/pods/af34555d-dfb6-4130-93b4-6e969b9e62f6/volumes" Jul 10 18:44:41.824928 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:41.824900 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:41.826107 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:41.826083 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:52.311208 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:52.311175 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd"] Jul 10 18:44:52.311711 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:52.311399 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" podUID="5a859741-03fa-4482-8eb7-a6a93a976ea8" containerName="istio-proxy" containerID="cri-o://9cb5d2946acaf2eac6dde31b0429cba5d847d7cad54efb3ee302fdf34fcf38fc" gracePeriod=30 Jul 10 18:44:55.580417 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.580391 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:55.667378 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.667281 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhrdl\" (UniqueName: \"kubernetes.io/projected/5a859741-03fa-4482-8eb7-a6a93a976ea8-kube-api-access-jhrdl\") pod \"5a859741-03fa-4482-8eb7-a6a93a976ea8\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " Jul 10 18:44:55.667378 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.667320 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-workload-certs\") pod \"5a859741-03fa-4482-8eb7-a6a93a976ea8\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " Jul 10 18:44:55.667378 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.667340 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-workload-socket\") pod \"5a859741-03fa-4482-8eb7-a6a93a976ea8\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " Jul 10 18:44:55.667378 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.667367 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-podinfo\") pod \"5a859741-03fa-4482-8eb7-a6a93a976ea8\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " Jul 10 18:44:55.667725 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.667407 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-envoy\") pod \"5a859741-03fa-4482-8eb7-a6a93a976ea8\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " Jul 10 18:44:55.667725 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.667430 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-token\") pod \"5a859741-03fa-4482-8eb7-a6a93a976ea8\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " Jul 10 18:44:55.667725 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.667456 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a859741-03fa-4482-8eb7-a6a93a976ea8-istiod-ca-cert\") pod \"5a859741-03fa-4482-8eb7-a6a93a976ea8\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " Jul 10 18:44:55.667725 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.667508 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-credential-socket\") pod \"5a859741-03fa-4482-8eb7-a6a93a976ea8\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " Jul 10 18:44:55.667725 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.667532 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-data\") pod \"5a859741-03fa-4482-8eb7-a6a93a976ea8\" (UID: \"5a859741-03fa-4482-8eb7-a6a93a976ea8\") " Jul 10 18:44:55.667725 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.667684 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5a859741-03fa-4482-8eb7-a6a93a976ea8" (UID: "5a859741-03fa-4482-8eb7-a6a93a976ea8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:44:55.668268 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.667725 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5a859741-03fa-4482-8eb7-a6a93a976ea8" (UID: "5a859741-03fa-4482-8eb7-a6a93a976ea8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:44:55.668268 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.667869 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-workload-certs\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:44:55.668268 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.667889 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-workload-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:44:55.668268 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.668056 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5a859741-03fa-4482-8eb7-a6a93a976ea8" (UID: "5a859741-03fa-4482-8eb7-a6a93a976ea8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:44:55.668268 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.668164 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a859741-03fa-4482-8eb7-a6a93a976ea8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5a859741-03fa-4482-8eb7-a6a93a976ea8" (UID: "5a859741-03fa-4482-8eb7-a6a93a976ea8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:44:55.668476 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.668437 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-data" (OuterVolumeSpecName: "istio-data") pod "5a859741-03fa-4482-8eb7-a6a93a976ea8" (UID: "5a859741-03fa-4482-8eb7-a6a93a976ea8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:44:55.669846 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.669808 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5a859741-03fa-4482-8eb7-a6a93a976ea8" (UID: "5a859741-03fa-4482-8eb7-a6a93a976ea8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:44:55.669954 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.669857 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a859741-03fa-4482-8eb7-a6a93a976ea8-kube-api-access-jhrdl" (OuterVolumeSpecName: "kube-api-access-jhrdl") pod "5a859741-03fa-4482-8eb7-a6a93a976ea8" (UID: "5a859741-03fa-4482-8eb7-a6a93a976ea8"). InnerVolumeSpecName "kube-api-access-jhrdl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:44:55.670235 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.670214 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5a859741-03fa-4482-8eb7-a6a93a976ea8" (UID: "5a859741-03fa-4482-8eb7-a6a93a976ea8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 18:44:55.670295 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.670236 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-token" (OuterVolumeSpecName: "istio-token") pod "5a859741-03fa-4482-8eb7-a6a93a976ea8" (UID: "5a859741-03fa-4482-8eb7-a6a93a976ea8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:44:55.768897 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.768848 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhrdl\" (UniqueName: \"kubernetes.io/projected/5a859741-03fa-4482-8eb7-a6a93a976ea8-kube-api-access-jhrdl\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:44:55.768897 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.768894 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-podinfo\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:44:55.768897 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.768907 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-envoy\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:44:55.768897 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.768914 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-token\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:44:55.768897 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.768922 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a859741-03fa-4482-8eb7-a6a93a976ea8-istiod-ca-cert\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:44:55.769223 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.768930 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-credential-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:44:55.769223 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.768938 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a859741-03fa-4482-8eb7-a6a93a976ea8-istio-data\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:44:55.905458 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.905414 2571 generic.go:356] "Generic (PLEG): container finished" podID="5a859741-03fa-4482-8eb7-a6a93a976ea8" containerID="9cb5d2946acaf2eac6dde31b0429cba5d847d7cad54efb3ee302fdf34fcf38fc" exitCode=0 Jul 10 18:44:55.905709 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.905474 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" Jul 10 18:44:55.905709 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.905498 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" event={"ID":"5a859741-03fa-4482-8eb7-a6a93a976ea8","Type":"ContainerDied","Data":"9cb5d2946acaf2eac6dde31b0429cba5d847d7cad54efb3ee302fdf34fcf38fc"} Jul 10 18:44:55.905709 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.905593 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd" event={"ID":"5a859741-03fa-4482-8eb7-a6a93a976ea8","Type":"ContainerDied","Data":"1573442b2b0cad38a7206d4ecc1fbc6b8b19f3aa29a146fa4ec1e06052b9f454"} Jul 10 18:44:55.905709 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.905644 2571 scope.go:117] "RemoveContainer" containerID="9cb5d2946acaf2eac6dde31b0429cba5d847d7cad54efb3ee302fdf34fcf38fc" Jul 10 18:44:55.916881 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.916859 2571 scope.go:117] "RemoveContainer" containerID="9cb5d2946acaf2eac6dde31b0429cba5d847d7cad54efb3ee302fdf34fcf38fc" Jul 10 18:44:55.917216 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:44:55.917191 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cb5d2946acaf2eac6dde31b0429cba5d847d7cad54efb3ee302fdf34fcf38fc\": container with ID starting with 9cb5d2946acaf2eac6dde31b0429cba5d847d7cad54efb3ee302fdf34fcf38fc not found: ID does not exist" containerID="9cb5d2946acaf2eac6dde31b0429cba5d847d7cad54efb3ee302fdf34fcf38fc" Jul 10 18:44:55.917284 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.917229 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cb5d2946acaf2eac6dde31b0429cba5d847d7cad54efb3ee302fdf34fcf38fc"} err="failed to get container status \"9cb5d2946acaf2eac6dde31b0429cba5d847d7cad54efb3ee302fdf34fcf38fc\": rpc error: code = NotFound desc = could not find container \"9cb5d2946acaf2eac6dde31b0429cba5d847d7cad54efb3ee302fdf34fcf38fc\": container with ID starting with 9cb5d2946acaf2eac6dde31b0429cba5d847d7cad54efb3ee302fdf34fcf38fc not found: ID does not exist" Jul 10 18:44:55.923638 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.923548 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd"] Jul 10 18:44:55.930052 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:55.930021 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8dfa64c8-openshift-default-67c98b775-v8cgd"] Jul 10 18:44:57.684248 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:57.684216 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a859741-03fa-4482-8eb7-a6a93a976ea8" path="/var/lib/kubelet/pods/5a859741-03fa-4482-8eb7-a6a93a976ea8/volumes" Jul 10 18:44:58.341039 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.341005 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv"] Jul 10 18:44:58.341397 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.341384 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a859741-03fa-4482-8eb7-a6a93a976ea8" containerName="istio-proxy" Jul 10 18:44:58.341448 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.341399 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a859741-03fa-4482-8eb7-a6a93a976ea8" containerName="istio-proxy" Jul 10 18:44:58.341448 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.341422 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af34555d-dfb6-4130-93b4-6e969b9e62f6" containerName="istio-proxy" Jul 10 18:44:58.341448 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.341427 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="af34555d-dfb6-4130-93b4-6e969b9e62f6" containerName="istio-proxy" Jul 10 18:44:58.341544 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.341492 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="af34555d-dfb6-4130-93b4-6e969b9e62f6" containerName="istio-proxy" Jul 10 18:44:58.341544 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.341502 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a859741-03fa-4482-8eb7-a6a93a976ea8" containerName="istio-proxy" Jul 10 18:44:58.346496 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.346465 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.349607 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.349583 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-199d1e45-openshift-default-dockercfg-wxf8r\"" Jul 10 18:44:58.363485 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.363455 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv"] Jul 10 18:44:58.392043 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.392010 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b158054-a8df-4168-b870-45f39bccf174-istio-podinfo\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.392043 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.392051 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-istio-envoy\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.392283 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.392091 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b158054-a8df-4168-b870-45f39bccf174-istio-token\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.392283 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.392152 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b158054-a8df-4168-b870-45f39bccf174-istiod-ca-cert\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.392283 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.392195 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-istio-data\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.392283 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.392214 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twnlq\" (UniqueName: \"kubernetes.io/projected/7b158054-a8df-4168-b870-45f39bccf174-kube-api-access-twnlq\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.392283 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.392241 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-workload-certs\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.392467 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.392309 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-workload-socket\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.392467 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.392346 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-credential-socket\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.493029 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.492990 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b158054-a8df-4168-b870-45f39bccf174-istio-podinfo\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.493220 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.493037 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-istio-envoy\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.493220 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.493066 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b158054-a8df-4168-b870-45f39bccf174-istio-token\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.493220 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.493085 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b158054-a8df-4168-b870-45f39bccf174-istiod-ca-cert\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.493220 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.493200 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-istio-data\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.493437 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.493230 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twnlq\" (UniqueName: \"kubernetes.io/projected/7b158054-a8df-4168-b870-45f39bccf174-kube-api-access-twnlq\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.493437 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.493275 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-workload-certs\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.493437 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.493357 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-workload-socket\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.493437 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.493416 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-credential-socket\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.493762 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.493732 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-istio-data\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.493895 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.493861 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-workload-certs\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.493974 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.493894 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-workload-socket\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.493974 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.493935 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-credential-socket\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.494198 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.494176 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b158054-a8df-4168-b870-45f39bccf174-istiod-ca-cert\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.495879 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.495858 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-istio-envoy\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.496084 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.496057 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b158054-a8df-4168-b870-45f39bccf174-istio-podinfo\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.501980 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.501952 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b158054-a8df-4168-b870-45f39bccf174-istio-token\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.502097 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.502041 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twnlq\" (UniqueName: \"kubernetes.io/projected/7b158054-a8df-4168-b870-45f39bccf174-kube-api-access-twnlq\") pod \"e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.657396 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.657298 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:58.829279 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.829250 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv"] Jul 10 18:44:58.831445 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:44:58.831416 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b158054_a8df_4168_b870_45f39bccf174.slice/crio-7a958c85e6b35437a3b19714bff21f21071578243afdf39956c14486bc4c87d5 WatchSource:0}: Error finding container 7a958c85e6b35437a3b19714bff21f21071578243afdf39956c14486bc4c87d5: Status 404 returned error can't find the container with id 7a958c85e6b35437a3b19714bff21f21071578243afdf39956c14486bc4c87d5 Jul 10 18:44:58.833808 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.833765 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:44:58.833889 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.833873 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:44:58.833927 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.833915 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:44:58.926483 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.926380 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" event={"ID":"7b158054-a8df-4168-b870-45f39bccf174","Type":"ContainerStarted","Data":"d7a985edbbb53071f62c55722887e2f8fd5848ea9c114b152f35c3a63c1a0928"} Jul 10 18:44:58.926483 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.926449 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" event={"ID":"7b158054-a8df-4168-b870-45f39bccf174","Type":"ContainerStarted","Data":"7a958c85e6b35437a3b19714bff21f21071578243afdf39956c14486bc4c87d5"} Jul 10 18:44:58.952179 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:58.952118 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" podStartSLOduration=0.952102172 podStartE2EDuration="952.102172ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:44:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:44:58.949161714 +0000 UTC m=+1655.848430834" watchObservedRunningTime="2026-07-10 18:44:58.952102172 +0000 UTC m=+1655.851371279" Jul 10 18:44:59.658523 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:59.658478 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:59.663395 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:59.663366 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:59.932942 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:59.932854 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:44:59.933865 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:44:59.933846 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:45:04.262010 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:04.261965 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv"] Jul 10 18:45:04.262525 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:04.262258 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" podUID="7b158054-a8df-4168-b870-45f39bccf174" containerName="istio-proxy" containerID="cri-o://d7a985edbbb53071f62c55722887e2f8fd5848ea9c114b152f35c3a63c1a0928" gracePeriod=30 Jul 10 18:45:10.518754 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.518680 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:45:10.608533 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.608495 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-workload-certs\") pod \"7b158054-a8df-4168-b870-45f39bccf174\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " Jul 10 18:45:10.608533 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.608540 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b158054-a8df-4168-b870-45f39bccf174-istio-podinfo\") pod \"7b158054-a8df-4168-b870-45f39bccf174\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " Jul 10 18:45:10.608834 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.608594 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-istio-data\") pod \"7b158054-a8df-4168-b870-45f39bccf174\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " Jul 10 18:45:10.608834 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.608638 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-workload-socket\") pod \"7b158054-a8df-4168-b870-45f39bccf174\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " Jul 10 18:45:10.608834 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.608666 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twnlq\" (UniqueName: \"kubernetes.io/projected/7b158054-a8df-4168-b870-45f39bccf174-kube-api-access-twnlq\") pod \"7b158054-a8df-4168-b870-45f39bccf174\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " Jul 10 18:45:10.608834 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.608697 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b158054-a8df-4168-b870-45f39bccf174-istio-token\") pod \"7b158054-a8df-4168-b870-45f39bccf174\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " Jul 10 18:45:10.608834 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.608755 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-credential-socket\") pod \"7b158054-a8df-4168-b870-45f39bccf174\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " Jul 10 18:45:10.608834 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.608781 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-istio-envoy\") pod \"7b158054-a8df-4168-b870-45f39bccf174\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " Jul 10 18:45:10.608834 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.608817 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b158054-a8df-4168-b870-45f39bccf174-istiod-ca-cert\") pod \"7b158054-a8df-4168-b870-45f39bccf174\" (UID: \"7b158054-a8df-4168-b870-45f39bccf174\") " Jul 10 18:45:10.609203 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.609011 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7b158054-a8df-4168-b870-45f39bccf174" (UID: "7b158054-a8df-4168-b870-45f39bccf174"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:45:10.609203 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.609138 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7b158054-a8df-4168-b870-45f39bccf174" (UID: "7b158054-a8df-4168-b870-45f39bccf174"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:45:10.609297 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.609214 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7b158054-a8df-4168-b870-45f39bccf174" (UID: "7b158054-a8df-4168-b870-45f39bccf174"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:45:10.609365 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.609337 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b158054-a8df-4168-b870-45f39bccf174-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7b158054-a8df-4168-b870-45f39bccf174" (UID: "7b158054-a8df-4168-b870-45f39bccf174"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:45:10.609419 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.609406 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-istio-data" (OuterVolumeSpecName: "istio-data") pod "7b158054-a8df-4168-b870-45f39bccf174" (UID: "7b158054-a8df-4168-b870-45f39bccf174"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:45:10.611033 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.610997 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b158054-a8df-4168-b870-45f39bccf174-kube-api-access-twnlq" (OuterVolumeSpecName: "kube-api-access-twnlq") pod "7b158054-a8df-4168-b870-45f39bccf174" (UID: "7b158054-a8df-4168-b870-45f39bccf174"). InnerVolumeSpecName "kube-api-access-twnlq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:45:10.611168 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.611125 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7b158054-a8df-4168-b870-45f39bccf174" (UID: "7b158054-a8df-4168-b870-45f39bccf174"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:45:10.611302 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.611282 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7b158054-a8df-4168-b870-45f39bccf174-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7b158054-a8df-4168-b870-45f39bccf174" (UID: "7b158054-a8df-4168-b870-45f39bccf174"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 18:45:10.611345 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.611283 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b158054-a8df-4168-b870-45f39bccf174-istio-token" (OuterVolumeSpecName: "istio-token") pod "7b158054-a8df-4168-b870-45f39bccf174" (UID: "7b158054-a8df-4168-b870-45f39bccf174"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:45:10.709917 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.709877 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-credential-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:45:10.709917 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.709912 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-istio-envoy\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:45:10.709917 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.709922 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b158054-a8df-4168-b870-45f39bccf174-istiod-ca-cert\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:45:10.710158 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.709930 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-workload-certs\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:45:10.710158 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.709938 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b158054-a8df-4168-b870-45f39bccf174-istio-podinfo\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:45:10.710158 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.709946 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-istio-data\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:45:10.710158 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.709954 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b158054-a8df-4168-b870-45f39bccf174-workload-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:45:10.710158 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.709961 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twnlq\" (UniqueName: \"kubernetes.io/projected/7b158054-a8df-4168-b870-45f39bccf174-kube-api-access-twnlq\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:45:10.710158 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.709971 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b158054-a8df-4168-b870-45f39bccf174-istio-token\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:45:10.995359 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.995326 2571 generic.go:356] "Generic (PLEG): container finished" podID="7b158054-a8df-4168-b870-45f39bccf174" containerID="d7a985edbbb53071f62c55722887e2f8fd5848ea9c114b152f35c3a63c1a0928" exitCode=0 Jul 10 18:45:10.995538 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.995370 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" Jul 10 18:45:10.995538 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.995408 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" event={"ID":"7b158054-a8df-4168-b870-45f39bccf174","Type":"ContainerDied","Data":"d7a985edbbb53071f62c55722887e2f8fd5848ea9c114b152f35c3a63c1a0928"} Jul 10 18:45:10.995538 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.995462 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv" event={"ID":"7b158054-a8df-4168-b870-45f39bccf174","Type":"ContainerDied","Data":"7a958c85e6b35437a3b19714bff21f21071578243afdf39956c14486bc4c87d5"} Jul 10 18:45:10.995538 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:10.995485 2571 scope.go:117] "RemoveContainer" containerID="d7a985edbbb53071f62c55722887e2f8fd5848ea9c114b152f35c3a63c1a0928" Jul 10 18:45:11.005196 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:11.005176 2571 scope.go:117] "RemoveContainer" containerID="d7a985edbbb53071f62c55722887e2f8fd5848ea9c114b152f35c3a63c1a0928" Jul 10 18:45:11.005477 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:45:11.005454 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7a985edbbb53071f62c55722887e2f8fd5848ea9c114b152f35c3a63c1a0928\": container with ID starting with d7a985edbbb53071f62c55722887e2f8fd5848ea9c114b152f35c3a63c1a0928 not found: ID does not exist" containerID="d7a985edbbb53071f62c55722887e2f8fd5848ea9c114b152f35c3a63c1a0928" Jul 10 18:45:11.005543 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:11.005490 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7a985edbbb53071f62c55722887e2f8fd5848ea9c114b152f35c3a63c1a0928"} err="failed to get container status \"d7a985edbbb53071f62c55722887e2f8fd5848ea9c114b152f35c3a63c1a0928\": rpc error: code = NotFound desc = could not find container \"d7a985edbbb53071f62c55722887e2f8fd5848ea9c114b152f35c3a63c1a0928\": container with ID starting with d7a985edbbb53071f62c55722887e2f8fd5848ea9c114b152f35c3a63c1a0928 not found: ID does not exist" Jul 10 18:45:11.017957 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:11.017927 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv"] Jul 10 18:45:11.026661 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:11.026631 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-199d1e45-openshift-default-756bb57d87-w6pcv"] Jul 10 18:45:11.683537 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:45:11.683500 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b158054-a8df-4168-b870-45f39bccf174" path="/var/lib/kubelet/pods/7b158054-a8df-4168-b870-45f39bccf174/volumes" Jul 10 18:46:47.156539 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.156504 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl"] Jul 10 18:46:47.157053 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.156894 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b158054-a8df-4168-b870-45f39bccf174" containerName="istio-proxy" Jul 10 18:46:47.157053 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.156907 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b158054-a8df-4168-b870-45f39bccf174" containerName="istio-proxy" Jul 10 18:46:47.157053 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.156965 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b158054-a8df-4168-b870-45f39bccf174" containerName="istio-proxy" Jul 10 18:46:47.159959 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.159939 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.163174 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.163147 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7f824989-openshift-default-dockercfg-lb4kh\"" Jul 10 18:46:47.178564 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.178533 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl"] Jul 10 18:46:47.253400 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.253361 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-workload-certs\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.253602 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.253412 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-data\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.253602 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.253485 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lt7q\" (UniqueName: \"kubernetes.io/projected/483ef7ce-58af-437a-9e25-56eec061eb6d-kube-api-access-8lt7q\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.253602 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.253516 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-workload-socket\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.253602 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.253535 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-podinfo\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.253602 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.253558 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-envoy\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.253602 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.253580 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-token\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.253865 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.253747 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-credential-socket\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.253865 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.253788 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/483ef7ce-58af-437a-9e25-56eec061eb6d-istiod-ca-cert\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.355215 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.355174 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lt7q\" (UniqueName: \"kubernetes.io/projected/483ef7ce-58af-437a-9e25-56eec061eb6d-kube-api-access-8lt7q\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.355414 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.355224 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-workload-socket\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.355414 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.355252 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-podinfo\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.355414 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.355285 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-envoy\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.355414 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.355314 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-token\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.355414 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.355361 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-credential-socket\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.355414 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.355382 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/483ef7ce-58af-437a-9e25-56eec061eb6d-istiod-ca-cert\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.355773 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.355431 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-workload-certs\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.355773 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.355468 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-data\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.355773 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.355671 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-workload-socket\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.355773 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.355749 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-credential-socket\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.355934 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.355839 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-data\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.355934 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.355883 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-workload-certs\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.356137 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.356114 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/483ef7ce-58af-437a-9e25-56eec061eb6d-istiod-ca-cert\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.357655 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.357609 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-envoy\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.358220 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.358198 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-podinfo\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.367987 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.367964 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-token\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.368088 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.368017 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lt7q\" (UniqueName: \"kubernetes.io/projected/483ef7ce-58af-437a-9e25-56eec061eb6d-kube-api-access-8lt7q\") pod \"e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.474894 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.474794 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:47.624659 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.624628 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl"] Jul 10 18:46:47.626354 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:46:47.626322 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod483ef7ce_58af_437a_9e25_56eec061eb6d.slice/crio-318ea46d2e2231e886f183f85674e1f21cfa49a2d3f85ef3395024007e2e7f31 WatchSource:0}: Error finding container 318ea46d2e2231e886f183f85674e1f21cfa49a2d3f85ef3395024007e2e7f31: Status 404 returned error can't find the container with id 318ea46d2e2231e886f183f85674e1f21cfa49a2d3f85ef3395024007e2e7f31 Jul 10 18:46:47.628476 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.628446 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:46:47.628547 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.628515 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:46:47.628547 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:47.628543 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:46:48.498687 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:48.498650 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" event={"ID":"483ef7ce-58af-437a-9e25-56eec061eb6d","Type":"ContainerStarted","Data":"feb785463cacc5062c47486be1ffba205ce12c6319aba97a97b38468a56b6dd9"} Jul 10 18:46:48.499140 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:48.498702 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" event={"ID":"483ef7ce-58af-437a-9e25-56eec061eb6d","Type":"ContainerStarted","Data":"318ea46d2e2231e886f183f85674e1f21cfa49a2d3f85ef3395024007e2e7f31"} Jul 10 18:46:48.521969 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:48.521906 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" podStartSLOduration=1.521885366 podStartE2EDuration="1.521885366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:46:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:46:48.520456515 +0000 UTC m=+1765.419725641" watchObservedRunningTime="2026-07-10 18:46:48.521885366 +0000 UTC m=+1765.421154472" Jul 10 18:46:49.475336 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:49.475289 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:49.480559 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:49.480533 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:49.504594 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:49.504556 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:46:49.505736 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:46:49.505719 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:47:04.506702 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:04.506665 2571 patch_prober.go:28] interesting pod/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 10 18:47:04.507149 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:04.506734 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" podUID="483ef7ce-58af-437a-9e25-56eec061eb6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 10 18:47:04.587464 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:04.587426 2571 generic.go:356] "Generic (PLEG): container finished" podID="483ef7ce-58af-437a-9e25-56eec061eb6d" containerID="feb785463cacc5062c47486be1ffba205ce12c6319aba97a97b38468a56b6dd9" exitCode=0 Jul 10 18:47:04.587679 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:04.587498 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" event={"ID":"483ef7ce-58af-437a-9e25-56eec061eb6d","Type":"ContainerDied","Data":"feb785463cacc5062c47486be1ffba205ce12c6319aba97a97b38468a56b6dd9"} Jul 10 18:47:04.587952 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:04.587938 2571 scope.go:117] "RemoveContainer" containerID="feb785463cacc5062c47486be1ffba205ce12c6319aba97a97b38468a56b6dd9" Jul 10 18:47:04.588959 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:04.588934 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:47:04.589012 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:04.589003 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:47:04.589048 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:04.589033 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:47:05.594796 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:05.594762 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" event={"ID":"483ef7ce-58af-437a-9e25-56eec061eb6d","Type":"ContainerStarted","Data":"57ef41a41d7318c467e6891157ee49754e783f9fe6d0975feef92eaf125b113b"} Jul 10 18:47:06.475722 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:06.475683 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:47:06.475928 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:06.475906 2571 patch_prober.go:28] interesting pod/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 10 18:47:06.475986 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:06.475967 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" podUID="483ef7ce-58af-437a-9e25-56eec061eb6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 10 18:47:07.475256 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:07.475218 2571 patch_prober.go:28] interesting pod/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 10 18:47:07.475727 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:07.475274 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" podUID="483ef7ce-58af-437a-9e25-56eec061eb6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 10 18:47:08.475409 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:08.475363 2571 patch_prober.go:28] interesting pod/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 10 18:47:08.475942 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:08.475441 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" podUID="483ef7ce-58af-437a-9e25-56eec061eb6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 10 18:47:09.120392 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:09.120352 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl"] Jul 10 18:47:09.121489 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:09.121221 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" podUID="483ef7ce-58af-437a-9e25-56eec061eb6d" containerName="istio-proxy" containerID="cri-o://57ef41a41d7318c467e6891157ee49754e783f9fe6d0975feef92eaf125b113b" gracePeriod=30 Jul 10 18:47:14.646301 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.646260 2571 generic.go:356] "Generic (PLEG): container finished" podID="483ef7ce-58af-437a-9e25-56eec061eb6d" containerID="57ef41a41d7318c467e6891157ee49754e783f9fe6d0975feef92eaf125b113b" exitCode=0 Jul 10 18:47:14.646758 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.646345 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" event={"ID":"483ef7ce-58af-437a-9e25-56eec061eb6d","Type":"ContainerDied","Data":"57ef41a41d7318c467e6891157ee49754e783f9fe6d0975feef92eaf125b113b"} Jul 10 18:47:14.646758 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.646437 2571 scope.go:117] "RemoveContainer" containerID="feb785463cacc5062c47486be1ffba205ce12c6319aba97a97b38468a56b6dd9" Jul 10 18:47:14.687653 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.687603 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:47:14.817838 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.817706 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-token\") pod \"483ef7ce-58af-437a-9e25-56eec061eb6d\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " Jul 10 18:47:14.817838 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.817758 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-envoy\") pod \"483ef7ce-58af-437a-9e25-56eec061eb6d\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " Jul 10 18:47:14.817838 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.817789 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-credential-socket\") pod \"483ef7ce-58af-437a-9e25-56eec061eb6d\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " Jul 10 18:47:14.817838 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.817816 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-workload-certs\") pod \"483ef7ce-58af-437a-9e25-56eec061eb6d\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " Jul 10 18:47:14.817838 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.817844 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-workload-socket\") pod \"483ef7ce-58af-437a-9e25-56eec061eb6d\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " Jul 10 18:47:14.818246 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.817883 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-data\") pod \"483ef7ce-58af-437a-9e25-56eec061eb6d\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " Jul 10 18:47:14.818246 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.817909 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/483ef7ce-58af-437a-9e25-56eec061eb6d-istiod-ca-cert\") pod \"483ef7ce-58af-437a-9e25-56eec061eb6d\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " Jul 10 18:47:14.818246 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.817947 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-podinfo\") pod \"483ef7ce-58af-437a-9e25-56eec061eb6d\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " Jul 10 18:47:14.818246 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.818020 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lt7q\" (UniqueName: \"kubernetes.io/projected/483ef7ce-58af-437a-9e25-56eec061eb6d-kube-api-access-8lt7q\") pod \"483ef7ce-58af-437a-9e25-56eec061eb6d\" (UID: \"483ef7ce-58af-437a-9e25-56eec061eb6d\") " Jul 10 18:47:14.818246 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.818140 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "483ef7ce-58af-437a-9e25-56eec061eb6d" (UID: "483ef7ce-58af-437a-9e25-56eec061eb6d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:47:14.818246 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.818156 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "483ef7ce-58af-437a-9e25-56eec061eb6d" (UID: "483ef7ce-58af-437a-9e25-56eec061eb6d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:47:14.818590 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.818411 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "483ef7ce-58af-437a-9e25-56eec061eb6d" (UID: "483ef7ce-58af-437a-9e25-56eec061eb6d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:47:14.818590 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.818544 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-credential-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:47:14.818590 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.818563 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-workload-certs\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:47:14.818590 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.818577 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-workload-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:47:14.819770 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.819735 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-data" (OuterVolumeSpecName: "istio-data") pod "483ef7ce-58af-437a-9e25-56eec061eb6d" (UID: "483ef7ce-58af-437a-9e25-56eec061eb6d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:47:14.819893 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.819744 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/483ef7ce-58af-437a-9e25-56eec061eb6d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "483ef7ce-58af-437a-9e25-56eec061eb6d" (UID: "483ef7ce-58af-437a-9e25-56eec061eb6d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:47:14.820282 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.820253 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-token" (OuterVolumeSpecName: "istio-token") pod "483ef7ce-58af-437a-9e25-56eec061eb6d" (UID: "483ef7ce-58af-437a-9e25-56eec061eb6d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:47:14.820481 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.820455 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "483ef7ce-58af-437a-9e25-56eec061eb6d" (UID: "483ef7ce-58af-437a-9e25-56eec061eb6d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 18:47:14.820564 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.820504 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "483ef7ce-58af-437a-9e25-56eec061eb6d" (UID: "483ef7ce-58af-437a-9e25-56eec061eb6d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:47:14.820602 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.820570 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/483ef7ce-58af-437a-9e25-56eec061eb6d-kube-api-access-8lt7q" (OuterVolumeSpecName: "kube-api-access-8lt7q") pod "483ef7ce-58af-437a-9e25-56eec061eb6d" (UID: "483ef7ce-58af-437a-9e25-56eec061eb6d"). InnerVolumeSpecName "kube-api-access-8lt7q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:47:14.919066 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.919007 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-data\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:47:14.919066 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.919057 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/483ef7ce-58af-437a-9e25-56eec061eb6d-istiod-ca-cert\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:47:14.919066 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.919071 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-podinfo\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:47:14.919066 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.919082 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lt7q\" (UniqueName: \"kubernetes.io/projected/483ef7ce-58af-437a-9e25-56eec061eb6d-kube-api-access-8lt7q\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:47:14.919356 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.919096 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-token\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:47:14.919356 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:14.919107 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/483ef7ce-58af-437a-9e25-56eec061eb6d-istio-envoy\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:47:15.653071 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:15.653040 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" Jul 10 18:47:15.653071 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:15.653050 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl" event={"ID":"483ef7ce-58af-437a-9e25-56eec061eb6d","Type":"ContainerDied","Data":"318ea46d2e2231e886f183f85674e1f21cfa49a2d3f85ef3395024007e2e7f31"} Jul 10 18:47:15.653571 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:15.653112 2571 scope.go:117] "RemoveContainer" containerID="57ef41a41d7318c467e6891157ee49754e783f9fe6d0975feef92eaf125b113b" Jul 10 18:47:15.685286 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:15.685255 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl"] Jul 10 18:47:15.685554 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:15.685534 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7f824989-openshift-default-8c7f8f6bc-hvfrl"] Jul 10 18:47:17.683933 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:17.683900 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="483ef7ce-58af-437a-9e25-56eec061eb6d" path="/var/lib/kubelet/pods/483ef7ce-58af-437a-9e25-56eec061eb6d/volumes" Jul 10 18:47:23.671073 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:23.671035 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r9vz_0cafad48-3998-465d-b4c8-b5d94cac6b3c/ovn-acl-logging/0.log" Jul 10 18:47:23.674349 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:23.674316 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r9vz_0cafad48-3998-465d-b4c8-b5d94cac6b3c/ovn-acl-logging/0.log" Jul 10 18:47:29.229649 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.229592 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4"] Jul 10 18:47:29.230148 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.230121 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="483ef7ce-58af-437a-9e25-56eec061eb6d" containerName="istio-proxy" Jul 10 18:47:29.230148 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.230139 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="483ef7ce-58af-437a-9e25-56eec061eb6d" containerName="istio-proxy" Jul 10 18:47:29.230274 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.230256 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="483ef7ce-58af-437a-9e25-56eec061eb6d" containerName="istio-proxy" Jul 10 18:47:29.230274 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.230269 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="483ef7ce-58af-437a-9e25-56eec061eb6d" containerName="istio-proxy" Jul 10 18:47:29.232574 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.232553 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.235420 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.235398 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-0a09b19d-openshift-default-dockercfg-qtd47\"" Jul 10 18:47:29.247546 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.247504 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a67eefee-3659-4238-8904-503a1de5b36f-istio-token\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.247546 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.247544 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-credential-socket\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.247855 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.247586 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58szj\" (UniqueName: \"kubernetes.io/projected/a67eefee-3659-4238-8904-503a1de5b36f-kube-api-access-58szj\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.247855 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.247638 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-workload-socket\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.247855 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.247667 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-istio-data\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.247855 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.247705 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-workload-certs\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.247855 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.247729 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a67eefee-3659-4238-8904-503a1de5b36f-istio-podinfo\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.247855 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.247755 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a67eefee-3659-4238-8904-503a1de5b36f-istiod-ca-cert\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.247855 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.247775 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-istio-envoy\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.248924 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.248897 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4"] Jul 10 18:47:29.348237 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.348194 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a67eefee-3659-4238-8904-503a1de5b36f-istio-token\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.348237 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.348243 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-credential-socket\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.348513 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.348288 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58szj\" (UniqueName: \"kubernetes.io/projected/a67eefee-3659-4238-8904-503a1de5b36f-kube-api-access-58szj\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.348513 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.348318 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-workload-socket\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.348513 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.348347 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-istio-data\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.348513 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.348393 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-workload-certs\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.348513 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.348419 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a67eefee-3659-4238-8904-503a1de5b36f-istio-podinfo\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.348513 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.348456 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a67eefee-3659-4238-8904-503a1de5b36f-istiod-ca-cert\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.348513 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.348491 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-istio-envoy\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.348879 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.348752 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-credential-socket\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.348879 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.348789 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-workload-socket\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.349369 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.349321 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-workload-certs\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.349555 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.349378 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-istio-data\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.349555 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.349489 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a67eefee-3659-4238-8904-503a1de5b36f-istiod-ca-cert\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.351107 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.351081 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-istio-envoy\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.351244 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.351214 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a67eefee-3659-4238-8904-503a1de5b36f-istio-podinfo\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.356700 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.356677 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a67eefee-3659-4238-8904-503a1de5b36f-istio-token\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.356823 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.356801 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58szj\" (UniqueName: \"kubernetes.io/projected/a67eefee-3659-4238-8904-503a1de5b36f-kube-api-access-58szj\") pod \"e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.543393 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.543349 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:29.705566 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.705544 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4"] Jul 10 18:47:29.707913 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:47:29.707879 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda67eefee_3659_4238_8904_503a1de5b36f.slice/crio-3509f4d0dc2b3a6ec6f4164780bf9a5cbd4bb3b659c85fa670535dce8e53a69c WatchSource:0}: Error finding container 3509f4d0dc2b3a6ec6f4164780bf9a5cbd4bb3b659c85fa670535dce8e53a69c: Status 404 returned error can't find the container with id 3509f4d0dc2b3a6ec6f4164780bf9a5cbd4bb3b659c85fa670535dce8e53a69c Jul 10 18:47:29.710491 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.710458 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:47:29.710590 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.710529 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:47:29.710590 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.710559 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:47:29.733188 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:29.733158 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" event={"ID":"a67eefee-3659-4238-8904-503a1de5b36f","Type":"ContainerStarted","Data":"3509f4d0dc2b3a6ec6f4164780bf9a5cbd4bb3b659c85fa670535dce8e53a69c"} Jul 10 18:47:30.740401 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:30.740359 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" event={"ID":"a67eefee-3659-4238-8904-503a1de5b36f","Type":"ContainerStarted","Data":"ec7a807e47228b204fbcb7e0ea874f43e8024af34b216416606822e58c8f1ac2"} Jul 10 18:47:30.763672 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:30.763608 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" podStartSLOduration=1.763591011 podStartE2EDuration="1.763591011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:47:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:47:30.760832791 +0000 UTC m=+1807.660101898" watchObservedRunningTime="2026-07-10 18:47:30.763591011 +0000 UTC m=+1807.662860118" Jul 10 18:47:31.543957 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:31.543917 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:31.548889 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:31.548865 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:31.745661 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:31.745602 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:31.746679 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:31.746653 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:47.786812 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:47.786766 2571 patch_prober.go:28] interesting pod/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 18:47:47.788282 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:47.786840 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" podUID="a67eefee-3659-4238-8904-503a1de5b36f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 18:47:48.834547 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:48.834512 2571 generic.go:356] "Generic (PLEG): container finished" podID="a67eefee-3659-4238-8904-503a1de5b36f" containerID="ec7a807e47228b204fbcb7e0ea874f43e8024af34b216416606822e58c8f1ac2" exitCode=0 Jul 10 18:47:48.835015 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:48.834582 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" event={"ID":"a67eefee-3659-4238-8904-503a1de5b36f","Type":"ContainerDied","Data":"ec7a807e47228b204fbcb7e0ea874f43e8024af34b216416606822e58c8f1ac2"} Jul 10 18:47:48.835015 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:48.834995 2571 scope.go:117] "RemoveContainer" containerID="ec7a807e47228b204fbcb7e0ea874f43e8024af34b216416606822e58c8f1ac2" Jul 10 18:47:48.836042 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:48.836013 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:47:48.836106 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:48.836087 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:47:48.836143 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:48.836117 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:47:49.847785 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:49.847747 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" event={"ID":"a67eefee-3659-4238-8904-503a1de5b36f","Type":"ContainerStarted","Data":"0ae43cb52d66b94619604db9bf444b5dcfd89016cf9ab852851e846fcbed70fa"} Jul 10 18:47:50.544385 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:50.544343 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:50.544649 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:50.544605 2571 patch_prober.go:28] interesting pod/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 10 18:47:50.544703 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:50.544677 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" podUID="a67eefee-3659-4238-8904-503a1de5b36f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 10 18:47:51.544833 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:51.544792 2571 patch_prober.go:28] interesting pod/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 10 18:47:51.545346 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:51.544861 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" podUID="a67eefee-3659-4238-8904-503a1de5b36f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 10 18:47:52.543849 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:52.543800 2571 patch_prober.go:28] interesting pod/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 10 18:47:52.544052 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:52.543881 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" podUID="a67eefee-3659-4238-8904-503a1de5b36f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 10 18:47:53.544531 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:53.544487 2571 patch_prober.go:28] interesting pod/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 10 18:47:53.544965 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:53.544558 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" podUID="a67eefee-3659-4238-8904-503a1de5b36f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 10 18:47:54.547194 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:54.547164 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:54.547624 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:54.547407 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:47:54.548068 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:47:54.548051 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:49:09.293959 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:09.293871 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4"] Jul 10 18:49:09.294492 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:09.294116 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" podUID="a67eefee-3659-4238-8904-503a1de5b36f" containerName="istio-proxy" containerID="cri-o://0ae43cb52d66b94619604db9bf444b5dcfd89016cf9ab852851e846fcbed70fa" gracePeriod=30 Jul 10 18:49:09.550595 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:09.550505 2571 patch_prober.go:28] interesting pod/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 18:49:09.550595 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:09.550568 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" podUID="a67eefee-3659-4238-8904-503a1de5b36f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 18:49:14.553255 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.553227 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:49:14.695186 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.695091 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58szj\" (UniqueName: \"kubernetes.io/projected/a67eefee-3659-4238-8904-503a1de5b36f-kube-api-access-58szj\") pod \"a67eefee-3659-4238-8904-503a1de5b36f\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " Jul 10 18:49:14.695186 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.695133 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a67eefee-3659-4238-8904-503a1de5b36f-istio-podinfo\") pod \"a67eefee-3659-4238-8904-503a1de5b36f\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " Jul 10 18:49:14.695186 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.695155 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a67eefee-3659-4238-8904-503a1de5b36f-istiod-ca-cert\") pod \"a67eefee-3659-4238-8904-503a1de5b36f\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " Jul 10 18:49:14.695484 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.695199 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-istio-data\") pod \"a67eefee-3659-4238-8904-503a1de5b36f\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " Jul 10 18:49:14.695484 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.695215 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-istio-envoy\") pod \"a67eefee-3659-4238-8904-503a1de5b36f\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " Jul 10 18:49:14.695484 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.695239 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-credential-socket\") pod \"a67eefee-3659-4238-8904-503a1de5b36f\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " Jul 10 18:49:14.695484 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.695282 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a67eefee-3659-4238-8904-503a1de5b36f-istio-token\") pod \"a67eefee-3659-4238-8904-503a1de5b36f\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " Jul 10 18:49:14.695484 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.695318 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-workload-certs\") pod \"a67eefee-3659-4238-8904-503a1de5b36f\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " Jul 10 18:49:14.695484 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.695358 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-workload-socket\") pod \"a67eefee-3659-4238-8904-503a1de5b36f\" (UID: \"a67eefee-3659-4238-8904-503a1de5b36f\") " Jul 10 18:49:14.695848 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.695704 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a67eefee-3659-4238-8904-503a1de5b36f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a67eefee-3659-4238-8904-503a1de5b36f" (UID: "a67eefee-3659-4238-8904-503a1de5b36f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:49:14.695848 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.695813 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a67eefee-3659-4238-8904-503a1de5b36f" (UID: "a67eefee-3659-4238-8904-503a1de5b36f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:49:14.695960 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.695902 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a67eefee-3659-4238-8904-503a1de5b36f" (UID: "a67eefee-3659-4238-8904-503a1de5b36f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:49:14.696016 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.695991 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a67eefee-3659-4238-8904-503a1de5b36f" (UID: "a67eefee-3659-4238-8904-503a1de5b36f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:49:14.696294 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.696265 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-istio-data" (OuterVolumeSpecName: "istio-data") pod "a67eefee-3659-4238-8904-503a1de5b36f" (UID: "a67eefee-3659-4238-8904-503a1de5b36f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:49:14.697485 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.697454 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a67eefee-3659-4238-8904-503a1de5b36f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a67eefee-3659-4238-8904-503a1de5b36f" (UID: "a67eefee-3659-4238-8904-503a1de5b36f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 18:49:14.697582 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.697494 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a67eefee-3659-4238-8904-503a1de5b36f-kube-api-access-58szj" (OuterVolumeSpecName: "kube-api-access-58szj") pod "a67eefee-3659-4238-8904-503a1de5b36f" (UID: "a67eefee-3659-4238-8904-503a1de5b36f"). InnerVolumeSpecName "kube-api-access-58szj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:49:14.699137 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.697529 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a67eefee-3659-4238-8904-503a1de5b36f-istio-token" (OuterVolumeSpecName: "istio-token") pod "a67eefee-3659-4238-8904-503a1de5b36f" (UID: "a67eefee-3659-4238-8904-503a1de5b36f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:49:14.699137 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.697533 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a67eefee-3659-4238-8904-503a1de5b36f" (UID: "a67eefee-3659-4238-8904-503a1de5b36f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:49:14.797100 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.797058 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-workload-certs\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:49:14.797100 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.797091 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-workload-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:49:14.797100 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.797100 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58szj\" (UniqueName: \"kubernetes.io/projected/a67eefee-3659-4238-8904-503a1de5b36f-kube-api-access-58szj\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:49:14.797100 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.797109 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a67eefee-3659-4238-8904-503a1de5b36f-istio-podinfo\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:49:14.797417 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.797117 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a67eefee-3659-4238-8904-503a1de5b36f-istiod-ca-cert\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:49:14.797417 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.797129 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-istio-data\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:49:14.797417 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.797140 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-istio-envoy\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:49:14.797417 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.797225 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a67eefee-3659-4238-8904-503a1de5b36f-credential-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:49:14.797417 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:14.797235 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a67eefee-3659-4238-8904-503a1de5b36f-istio-token\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:49:15.291287 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:15.291249 2571 generic.go:356] "Generic (PLEG): container finished" podID="a67eefee-3659-4238-8904-503a1de5b36f" containerID="0ae43cb52d66b94619604db9bf444b5dcfd89016cf9ab852851e846fcbed70fa" exitCode=0 Jul 10 18:49:15.291510 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:15.291313 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" Jul 10 18:49:15.291510 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:15.291328 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" event={"ID":"a67eefee-3659-4238-8904-503a1de5b36f","Type":"ContainerDied","Data":"0ae43cb52d66b94619604db9bf444b5dcfd89016cf9ab852851e846fcbed70fa"} Jul 10 18:49:15.291510 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:15.291410 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4" event={"ID":"a67eefee-3659-4238-8904-503a1de5b36f","Type":"ContainerDied","Data":"3509f4d0dc2b3a6ec6f4164780bf9a5cbd4bb3b659c85fa670535dce8e53a69c"} Jul 10 18:49:15.291510 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:15.291441 2571 scope.go:117] "RemoveContainer" containerID="0ae43cb52d66b94619604db9bf444b5dcfd89016cf9ab852851e846fcbed70fa" Jul 10 18:49:15.301379 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:15.301358 2571 scope.go:117] "RemoveContainer" containerID="ec7a807e47228b204fbcb7e0ea874f43e8024af34b216416606822e58c8f1ac2" Jul 10 18:49:15.310078 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:15.310059 2571 scope.go:117] "RemoveContainer" containerID="0ae43cb52d66b94619604db9bf444b5dcfd89016cf9ab852851e846fcbed70fa" Jul 10 18:49:15.310428 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:49:15.310404 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ae43cb52d66b94619604db9bf444b5dcfd89016cf9ab852851e846fcbed70fa\": container with ID starting with 0ae43cb52d66b94619604db9bf444b5dcfd89016cf9ab852851e846fcbed70fa not found: ID does not exist" containerID="0ae43cb52d66b94619604db9bf444b5dcfd89016cf9ab852851e846fcbed70fa" Jul 10 18:49:15.310477 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:15.310439 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ae43cb52d66b94619604db9bf444b5dcfd89016cf9ab852851e846fcbed70fa"} err="failed to get container status \"0ae43cb52d66b94619604db9bf444b5dcfd89016cf9ab852851e846fcbed70fa\": rpc error: code = NotFound desc = could not find container \"0ae43cb52d66b94619604db9bf444b5dcfd89016cf9ab852851e846fcbed70fa\": container with ID starting with 0ae43cb52d66b94619604db9bf444b5dcfd89016cf9ab852851e846fcbed70fa not found: ID does not exist" Jul 10 18:49:15.310477 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:15.310459 2571 scope.go:117] "RemoveContainer" containerID="ec7a807e47228b204fbcb7e0ea874f43e8024af34b216416606822e58c8f1ac2" Jul 10 18:49:15.310769 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:49:15.310749 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec7a807e47228b204fbcb7e0ea874f43e8024af34b216416606822e58c8f1ac2\": container with ID starting with ec7a807e47228b204fbcb7e0ea874f43e8024af34b216416606822e58c8f1ac2 not found: ID does not exist" containerID="ec7a807e47228b204fbcb7e0ea874f43e8024af34b216416606822e58c8f1ac2" Jul 10 18:49:15.310850 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:15.310772 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec7a807e47228b204fbcb7e0ea874f43e8024af34b216416606822e58c8f1ac2"} err="failed to get container status \"ec7a807e47228b204fbcb7e0ea874f43e8024af34b216416606822e58c8f1ac2\": rpc error: code = NotFound desc = could not find container \"ec7a807e47228b204fbcb7e0ea874f43e8024af34b216416606822e58c8f1ac2\": container with ID starting with ec7a807e47228b204fbcb7e0ea874f43e8024af34b216416606822e58c8f1ac2 not found: ID does not exist" Jul 10 18:49:15.313874 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:15.313852 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4"] Jul 10 18:49:15.322018 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:15.321990 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-0a09b19d-openshift-default-6d48895859-bljw4"] Jul 10 18:49:15.683645 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:49:15.683547 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a67eefee-3659-4238-8904-503a1de5b36f" path="/var/lib/kubelet/pods/a67eefee-3659-4238-8904-503a1de5b36f/volumes" Jul 10 18:50:04.210516 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.210480 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m"] Jul 10 18:50:04.211032 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.210878 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="483ef7ce-58af-437a-9e25-56eec061eb6d" containerName="istio-proxy" Jul 10 18:50:04.211032 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.210893 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="483ef7ce-58af-437a-9e25-56eec061eb6d" containerName="istio-proxy" Jul 10 18:50:04.211032 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.210903 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a67eefee-3659-4238-8904-503a1de5b36f" containerName="istio-proxy" Jul 10 18:50:04.211032 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.210910 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a67eefee-3659-4238-8904-503a1de5b36f" containerName="istio-proxy" Jul 10 18:50:04.211032 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.210926 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a67eefee-3659-4238-8904-503a1de5b36f" containerName="istio-proxy" Jul 10 18:50:04.211032 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.210932 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a67eefee-3659-4238-8904-503a1de5b36f" containerName="istio-proxy" Jul 10 18:50:04.211032 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.210983 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a67eefee-3659-4238-8904-503a1de5b36f" containerName="istio-proxy" Jul 10 18:50:04.211032 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.210991 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a67eefee-3659-4238-8904-503a1de5b36f" containerName="istio-proxy" Jul 10 18:50:04.214106 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.214082 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.218389 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.218364 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-bee2bc-openshift-default-dockercfg-sbk7t\"" Jul 10 18:50:04.240095 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.240065 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m"] Jul 10 18:50:04.329682 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.329640 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-envoy\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.329925 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.329698 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8gjs\" (UniqueName: \"kubernetes.io/projected/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-kube-api-access-x8gjs\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.329925 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.329759 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-credential-socket\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.329925 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.329778 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-workload-certs\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.329925 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.329893 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-data\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.330123 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.329949 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istiod-ca-cert\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.330123 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.329982 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-workload-socket\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.330123 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.330042 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-token\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.330123 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.330072 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-podinfo\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.431086 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.431045 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-envoy\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.431086 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.431094 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8gjs\" (UniqueName: \"kubernetes.io/projected/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-kube-api-access-x8gjs\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.431373 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.431226 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-credential-socket\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.431373 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.431270 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-workload-certs\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.431373 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.431334 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-data\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.431530 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.431386 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istiod-ca-cert\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.431530 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.431416 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-workload-socket\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.431530 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.431463 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-token\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.431530 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.431494 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-podinfo\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.431777 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.431680 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-credential-socket\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.431855 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.431823 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-data\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.431916 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.431865 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-workload-socket\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.432103 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.432080 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-workload-certs\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.432168 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.432135 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istiod-ca-cert\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.433663 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.433639 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-envoy\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.433957 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.433938 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-podinfo\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.446794 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.446766 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-token\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.447024 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.447002 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8gjs\" (UniqueName: \"kubernetes.io/projected/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-kube-api-access-x8gjs\") pod \"e2e-inf-a-bee2bc-openshift-default-855989675-njp8m\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.525853 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.525814 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:04.696310 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.696280 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m"] Jul 10 18:50:04.698148 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:50:04.698118 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa5b0c02_47d9_4a51_b161_635a9a7e8bf2.slice/crio-35aa7bc30dceeff879d1c8f55c85f41c6297c8794f57f169e07e75cb0d420652 WatchSource:0}: Error finding container 35aa7bc30dceeff879d1c8f55c85f41c6297c8794f57f169e07e75cb0d420652: Status 404 returned error can't find the container with id 35aa7bc30dceeff879d1c8f55c85f41c6297c8794f57f169e07e75cb0d420652 Jul 10 18:50:04.700337 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.700304 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:50:04.700396 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.700376 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:50:04.700435 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:04.700406 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:50:05.551428 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:05.551390 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" event={"ID":"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2","Type":"ContainerStarted","Data":"24dd55f30a6561273de47d581d8302f701477359ea1c9b574096d9ddce038489"} Jul 10 18:50:05.551847 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:05.551455 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" event={"ID":"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2","Type":"ContainerStarted","Data":"35aa7bc30dceeff879d1c8f55c85f41c6297c8794f57f169e07e75cb0d420652"} Jul 10 18:50:05.582364 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:05.582302 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" podStartSLOduration=1.582281702 podStartE2EDuration="1.582281702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:50:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:50:05.579982203 +0000 UTC m=+1962.479251313" watchObservedRunningTime="2026-07-10 18:50:05.582281702 +0000 UTC m=+1962.481550866" Jul 10 18:50:06.526431 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:06.526392 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:06.532036 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:06.532004 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:06.557293 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:06.557265 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:06.558166 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:06.558145 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:12.827636 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:12.827587 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt"] Jul 10 18:50:12.831189 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:12.831171 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:12.834361 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:12.834332 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-9a1ca1-openshift-default-dockercfg-dl2s4\"" Jul 10 18:50:12.851076 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:12.851048 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt"] Jul 10 18:50:13.014927 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.014890 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-workload-socket\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.014927 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.014929 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d98786f-e555-4312-aa09-41972baed0ef-istiod-ca-cert\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.015204 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.014955 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-522jz\" (UniqueName: \"kubernetes.io/projected/7d98786f-e555-4312-aa09-41972baed0ef-kube-api-access-522jz\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.015204 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.015048 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-istio-data\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.015204 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.015129 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-istio-envoy\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.015204 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.015157 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d98786f-e555-4312-aa09-41972baed0ef-istio-podinfo\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.015204 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.015184 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-credential-socket\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.015526 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.015241 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d98786f-e555-4312-aa09-41972baed0ef-istio-token\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.015526 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.015304 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-workload-certs\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.116213 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.116091 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-workload-socket\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.116213 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.116135 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d98786f-e555-4312-aa09-41972baed0ef-istiod-ca-cert\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.116461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.116251 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-522jz\" (UniqueName: \"kubernetes.io/projected/7d98786f-e555-4312-aa09-41972baed0ef-kube-api-access-522jz\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.116461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.116291 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-istio-data\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.116461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.116359 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-istio-envoy\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.116461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.116380 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d98786f-e555-4312-aa09-41972baed0ef-istio-podinfo\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.116461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.116412 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-credential-socket\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.116461 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.116448 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d98786f-e555-4312-aa09-41972baed0ef-istio-token\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.116783 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.116523 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-workload-certs\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.116783 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.116567 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-workload-socket\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.116783 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.116654 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-istio-data\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.117554 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.117527 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-credential-socket\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.117757 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.117719 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-workload-certs\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.117961 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.117938 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d98786f-e555-4312-aa09-41972baed0ef-istiod-ca-cert\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.118923 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.118901 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-istio-envoy\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.119474 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.119449 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d98786f-e555-4312-aa09-41972baed0ef-istio-podinfo\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.128027 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.128000 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d98786f-e555-4312-aa09-41972baed0ef-istio-token\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.130508 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.130483 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-522jz\" (UniqueName: \"kubernetes.io/projected/7d98786f-e555-4312-aa09-41972baed0ef-kube-api-access-522jz\") pod \"e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.141877 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.141850 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:13.293443 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.293416 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt"] Jul 10 18:50:13.295117 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:50:13.295093 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d98786f_e555_4312_aa09_41972baed0ef.slice/crio-20438d0efa1ecbe335135d51f57a4d34b0cebb6d95229f84f22183e02e9f0904 WatchSource:0}: Error finding container 20438d0efa1ecbe335135d51f57a4d34b0cebb6d95229f84f22183e02e9f0904: Status 404 returned error can't find the container with id 20438d0efa1ecbe335135d51f57a4d34b0cebb6d95229f84f22183e02e9f0904 Jul 10 18:50:13.297288 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.297258 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:50:13.297359 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.297327 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:50:13.297394 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.297357 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:50:13.593859 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.593821 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" event={"ID":"7d98786f-e555-4312-aa09-41972baed0ef","Type":"ContainerStarted","Data":"17cd475b4983390f83b4fd5501933cb810b20a0cfcafd95f5b475aa1eb86c40d"} Jul 10 18:50:13.594019 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.593879 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" event={"ID":"7d98786f-e555-4312-aa09-41972baed0ef","Type":"ContainerStarted","Data":"20438d0efa1ecbe335135d51f57a4d34b0cebb6d95229f84f22183e02e9f0904"} Jul 10 18:50:13.619963 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:13.619880 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" podStartSLOduration=1.619860435 podStartE2EDuration="1.619860435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:50:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:50:13.617572988 +0000 UTC m=+1970.516842097" watchObservedRunningTime="2026-07-10 18:50:13.619860435 +0000 UTC m=+1970.519129545" Jul 10 18:50:14.142681 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:14.142637 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:14.147793 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:14.147753 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:14.599269 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:14.599233 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:14.600229 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:14.600207 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:22.598835 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:22.598769 2571 patch_prober.go:28] interesting pod/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 18:50:22.599306 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:22.598857 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" podUID="aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 18:50:24.666137 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:24.666095 2571 generic.go:356] "Generic (PLEG): container finished" podID="aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" containerID="24dd55f30a6561273de47d581d8302f701477359ea1c9b574096d9ddce038489" exitCode=0 Jul 10 18:50:24.666581 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:24.666173 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" event={"ID":"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2","Type":"ContainerDied","Data":"24dd55f30a6561273de47d581d8302f701477359ea1c9b574096d9ddce038489"} Jul 10 18:50:24.666649 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:24.666596 2571 scope.go:117] "RemoveContainer" containerID="24dd55f30a6561273de47d581d8302f701477359ea1c9b574096d9ddce038489" Jul 10 18:50:24.667642 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:24.667601 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:50:24.667719 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:24.667706 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:50:24.667760 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:24.667751 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:50:25.673601 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:25.673555 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" event={"ID":"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2","Type":"ContainerStarted","Data":"e04ed38aca45a55fc71cf9e45c22519d7dd1430ee0c294089f065bd3ec50a64f"} Jul 10 18:50:26.526970 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:26.526851 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:26.527715 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:26.527587 2571 patch_prober.go:28] interesting pod/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 10 18:50:26.527930 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:26.527907 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" podUID="aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 10 18:50:27.526871 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:27.526824 2571 patch_prober.go:28] interesting pod/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 10 18:50:27.527400 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:27.526897 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" podUID="aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 10 18:50:28.527101 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:28.527060 2571 patch_prober.go:28] interesting pod/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 10 18:50:28.527758 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:28.527724 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" podUID="aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 10 18:50:29.526531 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:29.526488 2571 patch_prober.go:28] interesting pod/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 10 18:50:29.526875 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:29.526837 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" podUID="aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 10 18:50:30.530187 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:30.530156 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:30.530630 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:30.530454 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:30.531457 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:30.531431 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:50:32.715796 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:32.715687 2571 generic.go:356] "Generic (PLEG): container finished" podID="7d98786f-e555-4312-aa09-41972baed0ef" containerID="17cd475b4983390f83b4fd5501933cb810b20a0cfcafd95f5b475aa1eb86c40d" exitCode=0 Jul 10 18:50:32.715796 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:32.715757 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" event={"ID":"7d98786f-e555-4312-aa09-41972baed0ef","Type":"ContainerDied","Data":"17cd475b4983390f83b4fd5501933cb810b20a0cfcafd95f5b475aa1eb86c40d"} Jul 10 18:50:32.716234 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:32.716197 2571 scope.go:117] "RemoveContainer" containerID="17cd475b4983390f83b4fd5501933cb810b20a0cfcafd95f5b475aa1eb86c40d" Jul 10 18:50:32.717239 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:32.717211 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:50:32.717302 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:32.717282 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:50:32.717337 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:32.717318 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:50:33.727884 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:33.727843 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" event={"ID":"7d98786f-e555-4312-aa09-41972baed0ef","Type":"ContainerStarted","Data":"e707395edadabbe8bfb4ce0ccb49c732898f7c82ae973a8ef711b6aa9096bd20"} Jul 10 18:50:34.142156 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:34.142111 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:34.142564 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:34.142534 2571 patch_prober.go:28] interesting pod/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 10 18:50:34.142667 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:34.142644 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" podUID="7d98786f-e555-4312-aa09-41972baed0ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 10 18:50:35.143075 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:35.143030 2571 patch_prober.go:28] interesting pod/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 10 18:50:35.143538 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:35.143092 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" podUID="7d98786f-e555-4312-aa09-41972baed0ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 10 18:50:36.143321 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:36.143282 2571 patch_prober.go:28] interesting pod/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 10 18:50:36.143754 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:36.143339 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" podUID="7d98786f-e555-4312-aa09-41972baed0ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 10 18:50:37.142662 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:37.142586 2571 patch_prober.go:28] interesting pod/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 10 18:50:37.142896 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:37.142707 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" podUID="7d98786f-e555-4312-aa09-41972baed0ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 10 18:50:38.145849 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:38.145813 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:38.146323 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:38.146110 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:50:38.147018 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:50:38.146997 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:52:23.705018 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:23.704985 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r9vz_0cafad48-3998-465d-b4c8-b5d94cac6b3c/ovn-acl-logging/0.log" Jul 10 18:52:23.708559 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:23.708540 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r9vz_0cafad48-3998-465d-b4c8-b5d94cac6b3c/ovn-acl-logging/0.log" Jul 10 18:52:28.833231 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:28.833188 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m"] Jul 10 18:52:28.833674 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:28.833401 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" podUID="aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" containerName="istio-proxy" containerID="cri-o://e04ed38aca45a55fc71cf9e45c22519d7dd1430ee0c294089f065bd3ec50a64f" gracePeriod=30 Jul 10 18:52:30.532579 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:30.532537 2571 patch_prober.go:28] interesting pod/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 18:52:30.532999 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:30.532609 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" podUID="aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 18:52:34.090831 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.090805 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:52:34.193709 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.193582 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-data\") pod \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " Jul 10 18:52:34.193709 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.193656 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-workload-socket\") pod \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " Jul 10 18:52:34.193709 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.193685 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-credential-socket\") pod \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " Jul 10 18:52:34.193979 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.193742 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-podinfo\") pod \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " Jul 10 18:52:34.193979 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.193820 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istiod-ca-cert\") pod \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " Jul 10 18:52:34.193979 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.193852 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8gjs\" (UniqueName: \"kubernetes.io/projected/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-kube-api-access-x8gjs\") pod \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " Jul 10 18:52:34.193979 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.193881 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-envoy\") pod \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " Jul 10 18:52:34.193979 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.193909 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-workload-certs\") pod \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " Jul 10 18:52:34.193979 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.193940 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-token\") pod \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\" (UID: \"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2\") " Jul 10 18:52:34.193979 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.193958 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" (UID: "aa5b0c02-47d9-4a51-b161-635a9a7e8bf2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:52:34.194334 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.194035 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" (UID: "aa5b0c02-47d9-4a51-b161-635a9a7e8bf2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:52:34.194334 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.194230 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" (UID: "aa5b0c02-47d9-4a51-b161-635a9a7e8bf2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:52:34.194334 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.194268 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" (UID: "aa5b0c02-47d9-4a51-b161-635a9a7e8bf2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:52:34.194334 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.194311 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istiod-ca-cert\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:52:34.194558 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.194334 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-workload-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:52:34.194558 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.194351 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-credential-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:52:34.194694 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.194595 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-data" (OuterVolumeSpecName: "istio-data") pod "aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" (UID: "aa5b0c02-47d9-4a51-b161-635a9a7e8bf2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:52:34.196166 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.196137 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" (UID: "aa5b0c02-47d9-4a51-b161-635a9a7e8bf2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 18:52:34.196430 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.196177 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-token" (OuterVolumeSpecName: "istio-token") pod "aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" (UID: "aa5b0c02-47d9-4a51-b161-635a9a7e8bf2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:52:34.196430 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.196194 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" (UID: "aa5b0c02-47d9-4a51-b161-635a9a7e8bf2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:52:34.196430 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.196227 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-kube-api-access-x8gjs" (OuterVolumeSpecName: "kube-api-access-x8gjs") pod "aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" (UID: "aa5b0c02-47d9-4a51-b161-635a9a7e8bf2"). InnerVolumeSpecName "kube-api-access-x8gjs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:52:34.294880 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.294845 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-data\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:52:34.294880 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.294876 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-podinfo\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:52:34.294880 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.294885 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8gjs\" (UniqueName: \"kubernetes.io/projected/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-kube-api-access-x8gjs\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:52:34.295115 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.294897 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-envoy\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:52:34.295115 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.294906 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-workload-certs\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:52:34.295115 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.294914 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2-istio-token\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:52:34.383705 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.383665 2571 generic.go:356] "Generic (PLEG): container finished" podID="aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" containerID="e04ed38aca45a55fc71cf9e45c22519d7dd1430ee0c294089f065bd3ec50a64f" exitCode=0 Jul 10 18:52:34.383881 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.383716 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" Jul 10 18:52:34.383881 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.383733 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" event={"ID":"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2","Type":"ContainerDied","Data":"e04ed38aca45a55fc71cf9e45c22519d7dd1430ee0c294089f065bd3ec50a64f"} Jul 10 18:52:34.383881 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.383787 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m" event={"ID":"aa5b0c02-47d9-4a51-b161-635a9a7e8bf2","Type":"ContainerDied","Data":"35aa7bc30dceeff879d1c8f55c85f41c6297c8794f57f169e07e75cb0d420652"} Jul 10 18:52:34.383881 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.383812 2571 scope.go:117] "RemoveContainer" containerID="e04ed38aca45a55fc71cf9e45c22519d7dd1430ee0c294089f065bd3ec50a64f" Jul 10 18:52:34.393592 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.393567 2571 scope.go:117] "RemoveContainer" containerID="24dd55f30a6561273de47d581d8302f701477359ea1c9b574096d9ddce038489" Jul 10 18:52:34.402099 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.402081 2571 scope.go:117] "RemoveContainer" containerID="e04ed38aca45a55fc71cf9e45c22519d7dd1430ee0c294089f065bd3ec50a64f" Jul 10 18:52:34.402383 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:52:34.402354 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e04ed38aca45a55fc71cf9e45c22519d7dd1430ee0c294089f065bd3ec50a64f\": container with ID starting with e04ed38aca45a55fc71cf9e45c22519d7dd1430ee0c294089f065bd3ec50a64f not found: ID does not exist" containerID="e04ed38aca45a55fc71cf9e45c22519d7dd1430ee0c294089f065bd3ec50a64f" Jul 10 18:52:34.402437 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.402394 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e04ed38aca45a55fc71cf9e45c22519d7dd1430ee0c294089f065bd3ec50a64f"} err="failed to get container status \"e04ed38aca45a55fc71cf9e45c22519d7dd1430ee0c294089f065bd3ec50a64f\": rpc error: code = NotFound desc = could not find container \"e04ed38aca45a55fc71cf9e45c22519d7dd1430ee0c294089f065bd3ec50a64f\": container with ID starting with e04ed38aca45a55fc71cf9e45c22519d7dd1430ee0c294089f065bd3ec50a64f not found: ID does not exist" Jul 10 18:52:34.402437 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.402413 2571 scope.go:117] "RemoveContainer" containerID="24dd55f30a6561273de47d581d8302f701477359ea1c9b574096d9ddce038489" Jul 10 18:52:34.402660 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:52:34.402639 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24dd55f30a6561273de47d581d8302f701477359ea1c9b574096d9ddce038489\": container with ID starting with 24dd55f30a6561273de47d581d8302f701477359ea1c9b574096d9ddce038489 not found: ID does not exist" containerID="24dd55f30a6561273de47d581d8302f701477359ea1c9b574096d9ddce038489" Jul 10 18:52:34.402708 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.402667 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24dd55f30a6561273de47d581d8302f701477359ea1c9b574096d9ddce038489"} err="failed to get container status \"24dd55f30a6561273de47d581d8302f701477359ea1c9b574096d9ddce038489\": rpc error: code = NotFound desc = could not find container \"24dd55f30a6561273de47d581d8302f701477359ea1c9b574096d9ddce038489\": container with ID starting with 24dd55f30a6561273de47d581d8302f701477359ea1c9b574096d9ddce038489 not found: ID does not exist" Jul 10 18:52:34.406587 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.406550 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m"] Jul 10 18:52:34.410533 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:34.410510 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-bee2bc-openshift-default-855989675-njp8m"] Jul 10 18:52:35.686928 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:35.686894 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" path="/var/lib/kubelet/pods/aa5b0c02-47d9-4a51-b161-635a9a7e8bf2/volumes" Jul 10 18:52:35.737389 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:35.737356 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt"] Jul 10 18:52:35.737599 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:35.737564 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" podUID="7d98786f-e555-4312-aa09-41972baed0ef" containerName="istio-proxy" containerID="cri-o://e707395edadabbe8bfb4ce0ccb49c732898f7c82ae973a8ef711b6aa9096bd20" gracePeriod=30 Jul 10 18:52:36.718961 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.718927 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6"] Jul 10 18:52:36.724964 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.724922 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" containerName="istio-proxy" Jul 10 18:52:36.725243 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.725218 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" containerName="istio-proxy" Jul 10 18:52:36.725309 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.725284 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" containerName="istio-proxy" Jul 10 18:52:36.725309 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.725295 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" containerName="istio-proxy" Jul 10 18:52:36.725561 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.725540 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" containerName="istio-proxy" Jul 10 18:52:36.725669 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.725568 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa5b0c02-47d9-4a51-b161-635a9a7e8bf2" containerName="istio-proxy" Jul 10 18:52:36.731083 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.731054 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.734312 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.734284 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-dc187f-openshift-default-dockercfg-82npm\"" Jul 10 18:52:36.747834 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.747801 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6"] Jul 10 18:52:36.815931 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.815807 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-workload-certs\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.815931 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.815861 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-workload-socket\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.815931 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.815903 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-credential-socket\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.815931 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.815934 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-envoy\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.816275 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.816051 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b980862e-40a0-4c8c-8b05-f3c322b397be-istiod-ca-cert\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.816275 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.816098 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-data\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.816275 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.816124 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z49hn\" (UniqueName: \"kubernetes.io/projected/b980862e-40a0-4c8c-8b05-f3c322b397be-kube-api-access-z49hn\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.816275 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.816200 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-podinfo\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.816275 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.816223 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-token\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.917045 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.917003 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-podinfo\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.917248 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.917054 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-token\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.917248 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.917136 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-workload-certs\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.917248 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.917167 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-workload-socket\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.917248 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.917204 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-credential-socket\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.917248 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.917236 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-envoy\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.917501 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.917298 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b980862e-40a0-4c8c-8b05-f3c322b397be-istiod-ca-cert\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.917501 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.917330 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-data\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.917501 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.917357 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z49hn\" (UniqueName: \"kubernetes.io/projected/b980862e-40a0-4c8c-8b05-f3c322b397be-kube-api-access-z49hn\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.917676 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.917648 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-workload-certs\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.917979 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.917951 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-workload-socket\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.918159 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.917968 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-credential-socket\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.918263 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.918048 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-data\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.918351 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.918303 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b980862e-40a0-4c8c-8b05-f3c322b397be-istiod-ca-cert\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.919994 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.919965 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-envoy\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.920413 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.920390 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-podinfo\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.928199 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.928169 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-token\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:36.930511 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:36.930481 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z49hn\" (UniqueName: \"kubernetes.io/projected/b980862e-40a0-4c8c-8b05-f3c322b397be-kube-api-access-z49hn\") pod \"e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:37.042115 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:37.042077 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:37.197097 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:37.197068 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6"] Jul 10 18:52:37.199965 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:52:37.199932 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb980862e_40a0_4c8c_8b05_f3c322b397be.slice/crio-4a7411f806487a24e38bd14079f712adb54c9aac6c4fddbd6d672b94636c81ad WatchSource:0}: Error finding container 4a7411f806487a24e38bd14079f712adb54c9aac6c4fddbd6d672b94636c81ad: Status 404 returned error can't find the container with id 4a7411f806487a24e38bd14079f712adb54c9aac6c4fddbd6d672b94636c81ad Jul 10 18:52:37.202117 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:37.202081 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:52:37.202231 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:37.202166 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:52:37.202231 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:37.202212 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:52:37.402756 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:37.402650 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" event={"ID":"b980862e-40a0-4c8c-8b05-f3c322b397be","Type":"ContainerStarted","Data":"3acca348baf5a4bed690a295a6d780c9432a44f027390c516ff8799e0376f6fd"} Jul 10 18:52:37.402756 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:37.402718 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" event={"ID":"b980862e-40a0-4c8c-8b05-f3c322b397be","Type":"ContainerStarted","Data":"4a7411f806487a24e38bd14079f712adb54c9aac6c4fddbd6d672b94636c81ad"} Jul 10 18:52:37.427108 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:37.427052 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" podStartSLOduration=1.427034243 podStartE2EDuration="1.427034243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:52:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:52:37.425224542 +0000 UTC m=+2114.324493651" watchObservedRunningTime="2026-07-10 18:52:37.427034243 +0000 UTC m=+2114.326303351" Jul 10 18:52:38.042482 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:38.042439 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:38.049203 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:38.049175 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:38.148591 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:38.148552 2571 patch_prober.go:28] interesting pod/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 18:52:38.148812 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:38.148641 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" podUID="7d98786f-e555-4312-aa09-41972baed0ef" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 18:52:38.408671 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:38.408550 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:38.409537 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:38.409521 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:41.000734 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.000666 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:52:41.158761 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.158721 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d98786f-e555-4312-aa09-41972baed0ef-istiod-ca-cert\") pod \"7d98786f-e555-4312-aa09-41972baed0ef\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " Jul 10 18:52:41.158978 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.158785 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d98786f-e555-4312-aa09-41972baed0ef-istio-podinfo\") pod \"7d98786f-e555-4312-aa09-41972baed0ef\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " Jul 10 18:52:41.158978 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.158809 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d98786f-e555-4312-aa09-41972baed0ef-istio-token\") pod \"7d98786f-e555-4312-aa09-41972baed0ef\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " Jul 10 18:52:41.158978 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.158924 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-workload-certs\") pod \"7d98786f-e555-4312-aa09-41972baed0ef\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " Jul 10 18:52:41.158978 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.158963 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-istio-data\") pod \"7d98786f-e555-4312-aa09-41972baed0ef\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " Jul 10 18:52:41.159207 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.159003 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-credential-socket\") pod \"7d98786f-e555-4312-aa09-41972baed0ef\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " Jul 10 18:52:41.159207 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.159047 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-workload-socket\") pod \"7d98786f-e555-4312-aa09-41972baed0ef\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " Jul 10 18:52:41.159207 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.159098 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-istio-envoy\") pod \"7d98786f-e555-4312-aa09-41972baed0ef\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " Jul 10 18:52:41.159207 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.159154 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7d98786f-e555-4312-aa09-41972baed0ef" (UID: "7d98786f-e555-4312-aa09-41972baed0ef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:52:41.159207 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.159166 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-522jz\" (UniqueName: \"kubernetes.io/projected/7d98786f-e555-4312-aa09-41972baed0ef-kube-api-access-522jz\") pod \"7d98786f-e555-4312-aa09-41972baed0ef\" (UID: \"7d98786f-e555-4312-aa09-41972baed0ef\") " Jul 10 18:52:41.159458 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.159193 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d98786f-e555-4312-aa09-41972baed0ef-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7d98786f-e555-4312-aa09-41972baed0ef" (UID: "7d98786f-e555-4312-aa09-41972baed0ef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:52:41.159458 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.159250 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7d98786f-e555-4312-aa09-41972baed0ef" (UID: "7d98786f-e555-4312-aa09-41972baed0ef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:52:41.159458 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.159311 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7d98786f-e555-4312-aa09-41972baed0ef" (UID: "7d98786f-e555-4312-aa09-41972baed0ef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:52:41.159650 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.159529 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d98786f-e555-4312-aa09-41972baed0ef-istiod-ca-cert\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:52:41.159650 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.159550 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-workload-certs\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:52:41.159650 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.159564 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-credential-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:52:41.159650 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.159578 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-workload-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:52:41.159974 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.159951 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-istio-data" (OuterVolumeSpecName: "istio-data") pod "7d98786f-e555-4312-aa09-41972baed0ef" (UID: "7d98786f-e555-4312-aa09-41972baed0ef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:52:41.161242 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.161211 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7d98786f-e555-4312-aa09-41972baed0ef" (UID: "7d98786f-e555-4312-aa09-41972baed0ef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:52:41.161423 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.161290 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7d98786f-e555-4312-aa09-41972baed0ef-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7d98786f-e555-4312-aa09-41972baed0ef" (UID: "7d98786f-e555-4312-aa09-41972baed0ef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 18:52:41.161525 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.161509 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d98786f-e555-4312-aa09-41972baed0ef-istio-token" (OuterVolumeSpecName: "istio-token") pod "7d98786f-e555-4312-aa09-41972baed0ef" (UID: "7d98786f-e555-4312-aa09-41972baed0ef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:52:41.161568 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.161529 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d98786f-e555-4312-aa09-41972baed0ef-kube-api-access-522jz" (OuterVolumeSpecName: "kube-api-access-522jz") pod "7d98786f-e555-4312-aa09-41972baed0ef" (UID: "7d98786f-e555-4312-aa09-41972baed0ef"). InnerVolumeSpecName "kube-api-access-522jz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:52:41.260121 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.260022 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-522jz\" (UniqueName: \"kubernetes.io/projected/7d98786f-e555-4312-aa09-41972baed0ef-kube-api-access-522jz\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:52:41.260121 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.260054 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d98786f-e555-4312-aa09-41972baed0ef-istio-podinfo\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:52:41.260121 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.260065 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d98786f-e555-4312-aa09-41972baed0ef-istio-token\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:52:41.260121 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.260076 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-istio-data\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:52:41.260121 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.260083 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d98786f-e555-4312-aa09-41972baed0ef-istio-envoy\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:52:41.427738 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.427700 2571 generic.go:356] "Generic (PLEG): container finished" podID="7d98786f-e555-4312-aa09-41972baed0ef" containerID="e707395edadabbe8bfb4ce0ccb49c732898f7c82ae973a8ef711b6aa9096bd20" exitCode=0 Jul 10 18:52:41.427916 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.427761 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" Jul 10 18:52:41.427916 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.427770 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" event={"ID":"7d98786f-e555-4312-aa09-41972baed0ef","Type":"ContainerDied","Data":"e707395edadabbe8bfb4ce0ccb49c732898f7c82ae973a8ef711b6aa9096bd20"} Jul 10 18:52:41.427916 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.427830 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt" event={"ID":"7d98786f-e555-4312-aa09-41972baed0ef","Type":"ContainerDied","Data":"20438d0efa1ecbe335135d51f57a4d34b0cebb6d95229f84f22183e02e9f0904"} Jul 10 18:52:41.427916 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.427855 2571 scope.go:117] "RemoveContainer" containerID="e707395edadabbe8bfb4ce0ccb49c732898f7c82ae973a8ef711b6aa9096bd20" Jul 10 18:52:41.437836 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.437815 2571 scope.go:117] "RemoveContainer" containerID="17cd475b4983390f83b4fd5501933cb810b20a0cfcafd95f5b475aa1eb86c40d" Jul 10 18:52:41.446532 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.446514 2571 scope.go:117] "RemoveContainer" containerID="e707395edadabbe8bfb4ce0ccb49c732898f7c82ae973a8ef711b6aa9096bd20" Jul 10 18:52:41.446968 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:52:41.446946 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e707395edadabbe8bfb4ce0ccb49c732898f7c82ae973a8ef711b6aa9096bd20\": container with ID starting with e707395edadabbe8bfb4ce0ccb49c732898f7c82ae973a8ef711b6aa9096bd20 not found: ID does not exist" containerID="e707395edadabbe8bfb4ce0ccb49c732898f7c82ae973a8ef711b6aa9096bd20" Jul 10 18:52:41.447042 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.446979 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e707395edadabbe8bfb4ce0ccb49c732898f7c82ae973a8ef711b6aa9096bd20"} err="failed to get container status \"e707395edadabbe8bfb4ce0ccb49c732898f7c82ae973a8ef711b6aa9096bd20\": rpc error: code = NotFound desc = could not find container \"e707395edadabbe8bfb4ce0ccb49c732898f7c82ae973a8ef711b6aa9096bd20\": container with ID starting with e707395edadabbe8bfb4ce0ccb49c732898f7c82ae973a8ef711b6aa9096bd20 not found: ID does not exist" Jul 10 18:52:41.447042 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.446998 2571 scope.go:117] "RemoveContainer" containerID="17cd475b4983390f83b4fd5501933cb810b20a0cfcafd95f5b475aa1eb86c40d" Jul 10 18:52:41.447286 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:52:41.447264 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17cd475b4983390f83b4fd5501933cb810b20a0cfcafd95f5b475aa1eb86c40d\": container with ID starting with 17cd475b4983390f83b4fd5501933cb810b20a0cfcafd95f5b475aa1eb86c40d not found: ID does not exist" containerID="17cd475b4983390f83b4fd5501933cb810b20a0cfcafd95f5b475aa1eb86c40d" Jul 10 18:52:41.447338 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.447294 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17cd475b4983390f83b4fd5501933cb810b20a0cfcafd95f5b475aa1eb86c40d"} err="failed to get container status \"17cd475b4983390f83b4fd5501933cb810b20a0cfcafd95f5b475aa1eb86c40d\": rpc error: code = NotFound desc = could not find container \"17cd475b4983390f83b4fd5501933cb810b20a0cfcafd95f5b475aa1eb86c40d\": container with ID starting with 17cd475b4983390f83b4fd5501933cb810b20a0cfcafd95f5b475aa1eb86c40d not found: ID does not exist" Jul 10 18:52:41.456744 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.456716 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt"] Jul 10 18:52:41.461019 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.460998 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-9a1ca1-openshift-default-6f54797cc5-9w5pt"] Jul 10 18:52:41.684372 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:41.684290 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d98786f-e555-4312-aa09-41972baed0ef" path="/var/lib/kubelet/pods/7d98786f-e555-4312-aa09-41972baed0ef/volumes" Jul 10 18:52:44.710988 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.710952 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6"] Jul 10 18:52:44.711556 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.711535 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d98786f-e555-4312-aa09-41972baed0ef" containerName="istio-proxy" Jul 10 18:52:44.711556 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.711557 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d98786f-e555-4312-aa09-41972baed0ef" containerName="istio-proxy" Jul 10 18:52:44.711733 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.711664 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d98786f-e555-4312-aa09-41972baed0ef" containerName="istio-proxy" Jul 10 18:52:44.711733 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.711676 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d98786f-e555-4312-aa09-41972baed0ef" containerName="istio-proxy" Jul 10 18:52:44.716407 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.716379 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.719030 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.718993 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-1d5607-openshift-default-dockercfg-fmtb6\"" Jul 10 18:52:44.726991 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.726959 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6"] Jul 10 18:52:44.794477 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.794436 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-data\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.794719 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.794512 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/258f6865-8c58-4609-85cb-2a8a2212dc2a-istiod-ca-cert\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.794719 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.794580 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-workload-socket\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.794719 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.794610 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-credential-socket\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.794719 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.794663 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-envoy\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.794946 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.794729 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-token\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.794946 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.794767 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-podinfo\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.794946 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.794801 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-workload-certs\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.794946 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.794832 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpqx4\" (UniqueName: \"kubernetes.io/projected/258f6865-8c58-4609-85cb-2a8a2212dc2a-kube-api-access-wpqx4\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.895844 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.895800 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/258f6865-8c58-4609-85cb-2a8a2212dc2a-istiod-ca-cert\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.895844 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.895849 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-workload-socket\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.896088 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.895874 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-credential-socket\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.896088 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.895922 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-envoy\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.896088 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.895952 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-token\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.896088 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.895981 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-podinfo\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.896088 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.896013 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-workload-certs\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.896349 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.896147 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpqx4\" (UniqueName: \"kubernetes.io/projected/258f6865-8c58-4609-85cb-2a8a2212dc2a-kube-api-access-wpqx4\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.896349 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.896207 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-data\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.896460 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.896442 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-workload-certs\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.896510 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.896460 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-credential-socket\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.896588 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.896569 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-workload-socket\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.896733 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.896715 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-data\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.896790 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.896742 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/258f6865-8c58-4609-85cb-2a8a2212dc2a-istiod-ca-cert\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.898325 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.898300 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-envoy\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.898489 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.898470 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-podinfo\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.904446 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.904417 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpqx4\" (UniqueName: \"kubernetes.io/projected/258f6865-8c58-4609-85cb-2a8a2212dc2a-kube-api-access-wpqx4\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:44.904562 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:44.904419 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-token\") pod \"e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:45.027868 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:45.027824 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:45.182733 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:45.182512 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6"] Jul 10 18:52:45.185745 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:52:45.185714 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod258f6865_8c58_4609_85cb_2a8a2212dc2a.slice/crio-ada2df0453f35c7ef738a778d5bfb37523e5c42a61eccdf6c91803da788b9543 WatchSource:0}: Error finding container ada2df0453f35c7ef738a778d5bfb37523e5c42a61eccdf6c91803da788b9543: Status 404 returned error can't find the container with id ada2df0453f35c7ef738a778d5bfb37523e5c42a61eccdf6c91803da788b9543 Jul 10 18:52:45.188204 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:45.188169 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:52:45.188282 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:45.188245 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:52:45.188282 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:45.188278 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:52:45.452883 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:45.452785 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" event={"ID":"258f6865-8c58-4609-85cb-2a8a2212dc2a","Type":"ContainerStarted","Data":"cd687191406afa4964612b2e7d7e0c809937b69402a0f0785d11d741bcfcdc6e"} Jul 10 18:52:45.452883 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:45.452847 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" event={"ID":"258f6865-8c58-4609-85cb-2a8a2212dc2a","Type":"ContainerStarted","Data":"ada2df0453f35c7ef738a778d5bfb37523e5c42a61eccdf6c91803da788b9543"} Jul 10 18:52:45.475680 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:45.475605 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" podStartSLOduration=1.4755850160000001 podStartE2EDuration="1.475585016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:52:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:52:45.473186213 +0000 UTC m=+2122.372455367" watchObservedRunningTime="2026-07-10 18:52:45.475585016 +0000 UTC m=+2122.374854124" Jul 10 18:52:46.028470 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:46.028430 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:46.033780 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:46.033754 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:46.457975 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:46.457892 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:46.458943 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:46.458921 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:52:54.450810 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:54.450765 2571 patch_prober.go:28] interesting pod/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 18:52:54.451267 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:54.450882 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" podUID="b980862e-40a0-4c8c-8b05-f3c322b397be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 18:52:55.507015 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:55.506982 2571 generic.go:356] "Generic (PLEG): container finished" podID="b980862e-40a0-4c8c-8b05-f3c322b397be" containerID="3acca348baf5a4bed690a295a6d780c9432a44f027390c516ff8799e0376f6fd" exitCode=0 Jul 10 18:52:55.507562 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:55.507057 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" event={"ID":"b980862e-40a0-4c8c-8b05-f3c322b397be","Type":"ContainerDied","Data":"3acca348baf5a4bed690a295a6d780c9432a44f027390c516ff8799e0376f6fd"} Jul 10 18:52:55.507669 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:55.507573 2571 scope.go:117] "RemoveContainer" containerID="3acca348baf5a4bed690a295a6d780c9432a44f027390c516ff8799e0376f6fd" Jul 10 18:52:55.508764 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:55.508737 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:52:55.508860 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:55.508845 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:52:55.508897 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:55.508887 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:52:56.516633 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:56.516574 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" event={"ID":"b980862e-40a0-4c8c-8b05-f3c322b397be","Type":"ContainerStarted","Data":"22d09efd1d4767a66aa74ccb6ec02a15d247aad5b9ec7bbc2e92de694ddaaaa1"} Jul 10 18:52:57.042552 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:57.042507 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:52:57.042857 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:57.042831 2571 patch_prober.go:28] interesting pod/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 10 18:52:57.042950 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:57.042894 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" podUID="b980862e-40a0-4c8c-8b05-f3c322b397be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 10 18:52:58.042564 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:58.042513 2571 patch_prober.go:28] interesting pod/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 10 18:52:58.043092 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:58.042590 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" podUID="b980862e-40a0-4c8c-8b05-f3c322b397be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 10 18:52:59.043502 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:59.043462 2571 patch_prober.go:28] interesting pod/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 10 18:52:59.043932 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:52:59.043575 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" podUID="b980862e-40a0-4c8c-8b05-f3c322b397be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 10 18:53:00.043431 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:00.043388 2571 patch_prober.go:28] interesting pod/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 10 18:53:00.043746 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:00.043712 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" podUID="b980862e-40a0-4c8c-8b05-f3c322b397be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 10 18:53:01.046282 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:01.046240 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:53:01.046776 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:01.046552 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:53:01.047479 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:01.047462 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:53:06.584739 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:06.584642 2571 generic.go:356] "Generic (PLEG): container finished" podID="258f6865-8c58-4609-85cb-2a8a2212dc2a" containerID="cd687191406afa4964612b2e7d7e0c809937b69402a0f0785d11d741bcfcdc6e" exitCode=0 Jul 10 18:53:06.584739 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:06.584705 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" event={"ID":"258f6865-8c58-4609-85cb-2a8a2212dc2a","Type":"ContainerDied","Data":"cd687191406afa4964612b2e7d7e0c809937b69402a0f0785d11d741bcfcdc6e"} Jul 10 18:53:06.585316 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:06.585253 2571 scope.go:117] "RemoveContainer" containerID="cd687191406afa4964612b2e7d7e0c809937b69402a0f0785d11d741bcfcdc6e" Jul 10 18:53:06.586367 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:06.586335 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:53:06.586435 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:06.586408 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:53:06.586472 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:06.586440 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:53:07.508265 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:07.508231 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6"] Jul 10 18:53:07.508490 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:07.508466 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" podUID="b980862e-40a0-4c8c-8b05-f3c322b397be" containerName="istio-proxy" containerID="cri-o://22d09efd1d4767a66aa74ccb6ec02a15d247aad5b9ec7bbc2e92de694ddaaaa1" gracePeriod=30 Jul 10 18:53:07.592037 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:07.592000 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" event={"ID":"258f6865-8c58-4609-85cb-2a8a2212dc2a","Type":"ContainerStarted","Data":"fb23206d65f36ea7aae1fd859437c5c3d97bae1a6cdeef44899a4e7fd08747fc"} Jul 10 18:53:08.028677 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:08.028638 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:53:08.028903 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:08.028784 2571 patch_prober.go:28] interesting pod/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 10 18:53:08.028903 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:08.028830 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" podUID="258f6865-8c58-4609-85cb-2a8a2212dc2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 10 18:53:09.028790 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:09.028741 2571 patch_prober.go:28] interesting pod/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 10 18:53:09.029328 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:09.028817 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" podUID="258f6865-8c58-4609-85cb-2a8a2212dc2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 10 18:53:10.028536 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:10.028485 2571 patch_prober.go:28] interesting pod/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 10 18:53:10.028779 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:10.028558 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" podUID="258f6865-8c58-4609-85cb-2a8a2212dc2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 10 18:53:11.029084 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:11.029034 2571 patch_prober.go:28] interesting pod/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 10 18:53:11.029638 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:11.029109 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" podUID="258f6865-8c58-4609-85cb-2a8a2212dc2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 10 18:53:12.032730 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.032696 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:53:12.033129 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.032968 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:53:12.033838 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.033819 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:53:12.775302 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.775230 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:53:12.874594 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.874560 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-workload-certs\") pod \"b980862e-40a0-4c8c-8b05-f3c322b397be\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " Jul 10 18:53:12.874594 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.874601 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-envoy\") pod \"b980862e-40a0-4c8c-8b05-f3c322b397be\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " Jul 10 18:53:12.874868 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.874652 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z49hn\" (UniqueName: \"kubernetes.io/projected/b980862e-40a0-4c8c-8b05-f3c322b397be-kube-api-access-z49hn\") pod \"b980862e-40a0-4c8c-8b05-f3c322b397be\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " Jul 10 18:53:12.874868 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.874694 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-podinfo\") pod \"b980862e-40a0-4c8c-8b05-f3c322b397be\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " Jul 10 18:53:12.874868 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.874739 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-data\") pod \"b980862e-40a0-4c8c-8b05-f3c322b397be\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " Jul 10 18:53:12.874868 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.874853 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b980862e-40a0-4c8c-8b05-f3c322b397be-istiod-ca-cert\") pod \"b980862e-40a0-4c8c-8b05-f3c322b397be\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " Jul 10 18:53:12.875079 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.874910 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-workload-socket\") pod \"b980862e-40a0-4c8c-8b05-f3c322b397be\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " Jul 10 18:53:12.875079 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.874919 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b980862e-40a0-4c8c-8b05-f3c322b397be" (UID: "b980862e-40a0-4c8c-8b05-f3c322b397be"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:53:12.875079 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.874957 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-credential-socket\") pod \"b980862e-40a0-4c8c-8b05-f3c322b397be\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " Jul 10 18:53:12.875079 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.874991 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-token\") pod \"b980862e-40a0-4c8c-8b05-f3c322b397be\" (UID: \"b980862e-40a0-4c8c-8b05-f3c322b397be\") " Jul 10 18:53:12.875293 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.875168 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b980862e-40a0-4c8c-8b05-f3c322b397be" (UID: "b980862e-40a0-4c8c-8b05-f3c322b397be"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:53:12.875293 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.875196 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b980862e-40a0-4c8c-8b05-f3c322b397be" (UID: "b980862e-40a0-4c8c-8b05-f3c322b397be"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:53:12.875293 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.875216 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b980862e-40a0-4c8c-8b05-f3c322b397be-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b980862e-40a0-4c8c-8b05-f3c322b397be" (UID: "b980862e-40a0-4c8c-8b05-f3c322b397be"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:53:12.875450 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.875343 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-credential-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:53:12.875450 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.875364 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-workload-certs\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:53:12.875450 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.875380 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b980862e-40a0-4c8c-8b05-f3c322b397be-istiod-ca-cert\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:53:12.875450 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.875394 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-workload-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:53:12.875754 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.875718 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-data" (OuterVolumeSpecName: "istio-data") pod "b980862e-40a0-4c8c-8b05-f3c322b397be" (UID: "b980862e-40a0-4c8c-8b05-f3c322b397be"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:53:12.876990 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.876969 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b980862e-40a0-4c8c-8b05-f3c322b397be" (UID: "b980862e-40a0-4c8c-8b05-f3c322b397be"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:53:12.877143 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.877123 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-token" (OuterVolumeSpecName: "istio-token") pod "b980862e-40a0-4c8c-8b05-f3c322b397be" (UID: "b980862e-40a0-4c8c-8b05-f3c322b397be"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:53:12.877271 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.877253 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b980862e-40a0-4c8c-8b05-f3c322b397be" (UID: "b980862e-40a0-4c8c-8b05-f3c322b397be"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 18:53:12.877327 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.877286 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b980862e-40a0-4c8c-8b05-f3c322b397be-kube-api-access-z49hn" (OuterVolumeSpecName: "kube-api-access-z49hn") pod "b980862e-40a0-4c8c-8b05-f3c322b397be" (UID: "b980862e-40a0-4c8c-8b05-f3c322b397be"). InnerVolumeSpecName "kube-api-access-z49hn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:53:12.975983 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.975948 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-token\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:53:12.975983 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.975977 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-envoy\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:53:12.975983 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.975986 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z49hn\" (UniqueName: \"kubernetes.io/projected/b980862e-40a0-4c8c-8b05-f3c322b397be-kube-api-access-z49hn\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:53:12.976241 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.975997 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-podinfo\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:53:12.976241 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:12.976007 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b980862e-40a0-4c8c-8b05-f3c322b397be-istio-data\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:53:13.626990 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:13.626952 2571 generic.go:356] "Generic (PLEG): container finished" podID="b980862e-40a0-4c8c-8b05-f3c322b397be" containerID="22d09efd1d4767a66aa74ccb6ec02a15d247aad5b9ec7bbc2e92de694ddaaaa1" exitCode=0 Jul 10 18:53:13.627437 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:13.627023 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" Jul 10 18:53:13.627437 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:13.627036 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" event={"ID":"b980862e-40a0-4c8c-8b05-f3c322b397be","Type":"ContainerDied","Data":"22d09efd1d4767a66aa74ccb6ec02a15d247aad5b9ec7bbc2e92de694ddaaaa1"} Jul 10 18:53:13.627437 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:13.627088 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6" event={"ID":"b980862e-40a0-4c8c-8b05-f3c322b397be","Type":"ContainerDied","Data":"4a7411f806487a24e38bd14079f712adb54c9aac6c4fddbd6d672b94636c81ad"} Jul 10 18:53:13.627437 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:13.627112 2571 scope.go:117] "RemoveContainer" containerID="22d09efd1d4767a66aa74ccb6ec02a15d247aad5b9ec7bbc2e92de694ddaaaa1" Jul 10 18:53:13.637012 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:13.636991 2571 scope.go:117] "RemoveContainer" containerID="3acca348baf5a4bed690a295a6d780c9432a44f027390c516ff8799e0376f6fd" Jul 10 18:53:13.645654 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:13.645634 2571 scope.go:117] "RemoveContainer" containerID="22d09efd1d4767a66aa74ccb6ec02a15d247aad5b9ec7bbc2e92de694ddaaaa1" Jul 10 18:53:13.645970 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:53:13.645949 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22d09efd1d4767a66aa74ccb6ec02a15d247aad5b9ec7bbc2e92de694ddaaaa1\": container with ID starting with 22d09efd1d4767a66aa74ccb6ec02a15d247aad5b9ec7bbc2e92de694ddaaaa1 not found: ID does not exist" containerID="22d09efd1d4767a66aa74ccb6ec02a15d247aad5b9ec7bbc2e92de694ddaaaa1" Jul 10 18:53:13.646032 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:13.645981 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22d09efd1d4767a66aa74ccb6ec02a15d247aad5b9ec7bbc2e92de694ddaaaa1"} err="failed to get container status \"22d09efd1d4767a66aa74ccb6ec02a15d247aad5b9ec7bbc2e92de694ddaaaa1\": rpc error: code = NotFound desc = could not find container \"22d09efd1d4767a66aa74ccb6ec02a15d247aad5b9ec7bbc2e92de694ddaaaa1\": container with ID starting with 22d09efd1d4767a66aa74ccb6ec02a15d247aad5b9ec7bbc2e92de694ddaaaa1 not found: ID does not exist" Jul 10 18:53:13.646032 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:13.646000 2571 scope.go:117] "RemoveContainer" containerID="3acca348baf5a4bed690a295a6d780c9432a44f027390c516ff8799e0376f6fd" Jul 10 18:53:13.646284 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:53:13.646269 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3acca348baf5a4bed690a295a6d780c9432a44f027390c516ff8799e0376f6fd\": container with ID starting with 3acca348baf5a4bed690a295a6d780c9432a44f027390c516ff8799e0376f6fd not found: ID does not exist" containerID="3acca348baf5a4bed690a295a6d780c9432a44f027390c516ff8799e0376f6fd" Jul 10 18:53:13.646338 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:13.646287 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3acca348baf5a4bed690a295a6d780c9432a44f027390c516ff8799e0376f6fd"} err="failed to get container status \"3acca348baf5a4bed690a295a6d780c9432a44f027390c516ff8799e0376f6fd\": rpc error: code = NotFound desc = could not find container \"3acca348baf5a4bed690a295a6d780c9432a44f027390c516ff8799e0376f6fd\": container with ID starting with 3acca348baf5a4bed690a295a6d780c9432a44f027390c516ff8799e0376f6fd not found: ID does not exist" Jul 10 18:53:13.653514 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:13.653488 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6"] Jul 10 18:53:13.662125 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:13.662099 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-dc187f-openshift-default-77678b6b9c-9xdc6"] Jul 10 18:53:13.684761 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:13.684735 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b980862e-40a0-4c8c-8b05-f3c322b397be" path="/var/lib/kubelet/pods/b980862e-40a0-4c8c-8b05-f3c322b397be/volumes" Jul 10 18:53:15.054192 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:15.054087 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6"] Jul 10 18:53:15.055111 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:15.055053 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" podUID="258f6865-8c58-4609-85cb-2a8a2212dc2a" containerName="istio-proxy" containerID="cri-o://fb23206d65f36ea7aae1fd859437c5c3d97bae1a6cdeef44899a4e7fd08747fc" gracePeriod=30 Jul 10 18:53:15.922311 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:15.922272 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g"] Jul 10 18:53:15.922840 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:15.922823 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b980862e-40a0-4c8c-8b05-f3c322b397be" containerName="istio-proxy" Jul 10 18:53:15.922906 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:15.922844 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b980862e-40a0-4c8c-8b05-f3c322b397be" containerName="istio-proxy" Jul 10 18:53:15.922906 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:15.922886 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d98786f-e555-4312-aa09-41972baed0ef" containerName="istio-proxy" Jul 10 18:53:15.922906 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:15.922894 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d98786f-e555-4312-aa09-41972baed0ef" containerName="istio-proxy" Jul 10 18:53:15.923023 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:15.923011 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b980862e-40a0-4c8c-8b05-f3c322b397be" containerName="istio-proxy" Jul 10 18:53:15.923058 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:15.923026 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b980862e-40a0-4c8c-8b05-f3c322b397be" containerName="istio-proxy" Jul 10 18:53:15.927411 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:15.927386 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:15.930197 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:15.930174 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-afdf1e-openshift-default-dockercfg-9zszk\"" Jul 10 18:53:15.938925 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:15.938889 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g"] Jul 10 18:53:16.001066 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.001020 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-credential-socket\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.001066 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.001063 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cswtr\" (UniqueName: \"kubernetes.io/projected/cccc2f13-7344-4b6f-9788-ef064d1dfde4-kube-api-access-cswtr\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.001306 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.001086 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-workload-socket\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.001306 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.001191 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-token\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.001306 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.001219 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-workload-certs\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.001306 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.001279 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-envoy\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.001306 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.001305 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-data\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.001485 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.001339 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-podinfo\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.001485 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.001364 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istiod-ca-cert\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.102436 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.102391 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-credential-socket\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.102436 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.102436 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cswtr\" (UniqueName: \"kubernetes.io/projected/cccc2f13-7344-4b6f-9788-ef064d1dfde4-kube-api-access-cswtr\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.103049 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.102459 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-workload-socket\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.103049 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.102493 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-token\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.103049 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.102522 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-workload-certs\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.103049 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.102557 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-envoy\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.103049 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.102684 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-data\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.103049 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.102771 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-podinfo\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.103049 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.102810 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istiod-ca-cert\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.103049 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.102926 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-credential-socket\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.103049 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.102973 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-workload-socket\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.103388 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.103044 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-workload-certs\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.103388 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.103130 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-data\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.103501 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.103481 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istiod-ca-cert\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.104933 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.104909 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-envoy\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.105232 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.105216 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-podinfo\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.111350 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.111316 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-token\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.111495 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.111435 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cswtr\" (UniqueName: \"kubernetes.io/projected/cccc2f13-7344-4b6f-9788-ef064d1dfde4-kube-api-access-cswtr\") pod \"e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.238836 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.238797 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:16.396293 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.396269 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g"] Jul 10 18:53:16.398499 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:53:16.398467 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcccc2f13_7344_4b6f_9788_ef064d1dfde4.slice/crio-6598d626cb2066f6a88668f662624778f96847ab43f10e0671858c5d0b3cbe58 WatchSource:0}: Error finding container 6598d626cb2066f6a88668f662624778f96847ab43f10e0671858c5d0b3cbe58: Status 404 returned error can't find the container with id 6598d626cb2066f6a88668f662624778f96847ab43f10e0671858c5d0b3cbe58 Jul 10 18:53:16.400815 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.400774 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:53:16.400887 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.400865 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:53:16.400934 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.400906 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:53:16.646912 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.646819 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" event={"ID":"cccc2f13-7344-4b6f-9788-ef064d1dfde4","Type":"ContainerStarted","Data":"31f82f9f4ca1e4f9108fa3263f814a507d0c3166f7eb9531acd6244184ac08a6"} Jul 10 18:53:16.646912 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.646873 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" event={"ID":"cccc2f13-7344-4b6f-9788-ef064d1dfde4","Type":"ContainerStarted","Data":"6598d626cb2066f6a88668f662624778f96847ab43f10e0671858c5d0b3cbe58"} Jul 10 18:53:16.669994 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:16.669939 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" podStartSLOduration=1.669917373 podStartE2EDuration="1.669917373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:53:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:53:16.668833941 +0000 UTC m=+2153.568103094" watchObservedRunningTime="2026-07-10 18:53:16.669917373 +0000 UTC m=+2153.569186481" Jul 10 18:53:17.239839 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:17.239801 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:17.245116 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:17.245087 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:17.652570 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:17.652474 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:17.653728 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:17.653709 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:20.327740 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.327668 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:53:20.442262 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.442217 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-podinfo\") pod \"258f6865-8c58-4609-85cb-2a8a2212dc2a\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " Jul 10 18:53:20.442262 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.442260 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-token\") pod \"258f6865-8c58-4609-85cb-2a8a2212dc2a\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " Jul 10 18:53:20.442509 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.442327 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-envoy\") pod \"258f6865-8c58-4609-85cb-2a8a2212dc2a\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " Jul 10 18:53:20.442509 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.442344 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpqx4\" (UniqueName: \"kubernetes.io/projected/258f6865-8c58-4609-85cb-2a8a2212dc2a-kube-api-access-wpqx4\") pod \"258f6865-8c58-4609-85cb-2a8a2212dc2a\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " Jul 10 18:53:20.442509 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.442381 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-data\") pod \"258f6865-8c58-4609-85cb-2a8a2212dc2a\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " Jul 10 18:53:20.442509 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.442412 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-workload-certs\") pod \"258f6865-8c58-4609-85cb-2a8a2212dc2a\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " Jul 10 18:53:20.442509 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.442444 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-credential-socket\") pod \"258f6865-8c58-4609-85cb-2a8a2212dc2a\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " Jul 10 18:53:20.442509 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.442475 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-workload-socket\") pod \"258f6865-8c58-4609-85cb-2a8a2212dc2a\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " Jul 10 18:53:20.442509 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.442502 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/258f6865-8c58-4609-85cb-2a8a2212dc2a-istiod-ca-cert\") pod \"258f6865-8c58-4609-85cb-2a8a2212dc2a\" (UID: \"258f6865-8c58-4609-85cb-2a8a2212dc2a\") " Jul 10 18:53:20.442987 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.442793 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "258f6865-8c58-4609-85cb-2a8a2212dc2a" (UID: "258f6865-8c58-4609-85cb-2a8a2212dc2a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:53:20.442987 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.442936 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-workload-certs\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:53:20.443154 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.443128 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/258f6865-8c58-4609-85cb-2a8a2212dc2a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "258f6865-8c58-4609-85cb-2a8a2212dc2a" (UID: "258f6865-8c58-4609-85cb-2a8a2212dc2a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:53:20.443443 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.443163 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "258f6865-8c58-4609-85cb-2a8a2212dc2a" (UID: "258f6865-8c58-4609-85cb-2a8a2212dc2a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:53:20.443443 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.443358 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "258f6865-8c58-4609-85cb-2a8a2212dc2a" (UID: "258f6865-8c58-4609-85cb-2a8a2212dc2a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:53:20.443443 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.443392 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-data" (OuterVolumeSpecName: "istio-data") pod "258f6865-8c58-4609-85cb-2a8a2212dc2a" (UID: "258f6865-8c58-4609-85cb-2a8a2212dc2a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:53:20.444810 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.444784 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-token" (OuterVolumeSpecName: "istio-token") pod "258f6865-8c58-4609-85cb-2a8a2212dc2a" (UID: "258f6865-8c58-4609-85cb-2a8a2212dc2a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:53:20.444901 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.444818 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/258f6865-8c58-4609-85cb-2a8a2212dc2a-kube-api-access-wpqx4" (OuterVolumeSpecName: "kube-api-access-wpqx4") pod "258f6865-8c58-4609-85cb-2a8a2212dc2a" (UID: "258f6865-8c58-4609-85cb-2a8a2212dc2a"). InnerVolumeSpecName "kube-api-access-wpqx4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:53:20.444901 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.444830 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "258f6865-8c58-4609-85cb-2a8a2212dc2a" (UID: "258f6865-8c58-4609-85cb-2a8a2212dc2a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:53:20.444901 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.444857 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "258f6865-8c58-4609-85cb-2a8a2212dc2a" (UID: "258f6865-8c58-4609-85cb-2a8a2212dc2a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 18:53:20.543629 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.543583 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-envoy\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:53:20.543816 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.543655 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpqx4\" (UniqueName: \"kubernetes.io/projected/258f6865-8c58-4609-85cb-2a8a2212dc2a-kube-api-access-wpqx4\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:53:20.543816 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.543666 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-data\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:53:20.543816 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.543674 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-credential-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:53:20.543816 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.543683 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/258f6865-8c58-4609-85cb-2a8a2212dc2a-workload-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:53:20.543816 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.543691 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/258f6865-8c58-4609-85cb-2a8a2212dc2a-istiod-ca-cert\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:53:20.543816 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.543699 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-podinfo\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:53:20.543816 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.543708 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/258f6865-8c58-4609-85cb-2a8a2212dc2a-istio-token\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:53:20.669367 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.669279 2571 generic.go:356] "Generic (PLEG): container finished" podID="258f6865-8c58-4609-85cb-2a8a2212dc2a" containerID="fb23206d65f36ea7aae1fd859437c5c3d97bae1a6cdeef44899a4e7fd08747fc" exitCode=0 Jul 10 18:53:20.669367 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.669333 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" Jul 10 18:53:20.669555 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.669364 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" event={"ID":"258f6865-8c58-4609-85cb-2a8a2212dc2a","Type":"ContainerDied","Data":"fb23206d65f36ea7aae1fd859437c5c3d97bae1a6cdeef44899a4e7fd08747fc"} Jul 10 18:53:20.669555 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.669415 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6" event={"ID":"258f6865-8c58-4609-85cb-2a8a2212dc2a","Type":"ContainerDied","Data":"ada2df0453f35c7ef738a778d5bfb37523e5c42a61eccdf6c91803da788b9543"} Jul 10 18:53:20.669555 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.669440 2571 scope.go:117] "RemoveContainer" containerID="fb23206d65f36ea7aae1fd859437c5c3d97bae1a6cdeef44899a4e7fd08747fc" Jul 10 18:53:20.678809 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.678794 2571 scope.go:117] "RemoveContainer" containerID="cd687191406afa4964612b2e7d7e0c809937b69402a0f0785d11d741bcfcdc6e" Jul 10 18:53:20.687147 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.687130 2571 scope.go:117] "RemoveContainer" containerID="fb23206d65f36ea7aae1fd859437c5c3d97bae1a6cdeef44899a4e7fd08747fc" Jul 10 18:53:20.687410 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:53:20.687392 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb23206d65f36ea7aae1fd859437c5c3d97bae1a6cdeef44899a4e7fd08747fc\": container with ID starting with fb23206d65f36ea7aae1fd859437c5c3d97bae1a6cdeef44899a4e7fd08747fc not found: ID does not exist" containerID="fb23206d65f36ea7aae1fd859437c5c3d97bae1a6cdeef44899a4e7fd08747fc" Jul 10 18:53:20.687480 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.687417 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb23206d65f36ea7aae1fd859437c5c3d97bae1a6cdeef44899a4e7fd08747fc"} err="failed to get container status \"fb23206d65f36ea7aae1fd859437c5c3d97bae1a6cdeef44899a4e7fd08747fc\": rpc error: code = NotFound desc = could not find container \"fb23206d65f36ea7aae1fd859437c5c3d97bae1a6cdeef44899a4e7fd08747fc\": container with ID starting with fb23206d65f36ea7aae1fd859437c5c3d97bae1a6cdeef44899a4e7fd08747fc not found: ID does not exist" Jul 10 18:53:20.687480 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.687432 2571 scope.go:117] "RemoveContainer" containerID="cd687191406afa4964612b2e7d7e0c809937b69402a0f0785d11d741bcfcdc6e" Jul 10 18:53:20.687718 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:53:20.687700 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd687191406afa4964612b2e7d7e0c809937b69402a0f0785d11d741bcfcdc6e\": container with ID starting with cd687191406afa4964612b2e7d7e0c809937b69402a0f0785d11d741bcfcdc6e not found: ID does not exist" containerID="cd687191406afa4964612b2e7d7e0c809937b69402a0f0785d11d741bcfcdc6e" Jul 10 18:53:20.687766 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.687724 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd687191406afa4964612b2e7d7e0c809937b69402a0f0785d11d741bcfcdc6e"} err="failed to get container status \"cd687191406afa4964612b2e7d7e0c809937b69402a0f0785d11d741bcfcdc6e\": rpc error: code = NotFound desc = could not find container \"cd687191406afa4964612b2e7d7e0c809937b69402a0f0785d11d741bcfcdc6e\": container with ID starting with cd687191406afa4964612b2e7d7e0c809937b69402a0f0785d11d741bcfcdc6e not found: ID does not exist" Jul 10 18:53:20.692571 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.692547 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6"] Jul 10 18:53:20.698160 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:20.698136 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-1d5607-openshift-default-d44f75b5f-wr2r6"] Jul 10 18:53:21.683328 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:21.683294 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="258f6865-8c58-4609-85cb-2a8a2212dc2a" path="/var/lib/kubelet/pods/258f6865-8c58-4609-85cb-2a8a2212dc2a/volumes" Jul 10 18:53:23.789062 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.789020 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4"] Jul 10 18:53:23.790688 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.790645 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="258f6865-8c58-4609-85cb-2a8a2212dc2a" containerName="istio-proxy" Jul 10 18:53:23.790688 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.790682 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="258f6865-8c58-4609-85cb-2a8a2212dc2a" containerName="istio-proxy" Jul 10 18:53:23.790849 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.790699 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b980862e-40a0-4c8c-8b05-f3c322b397be" containerName="istio-proxy" Jul 10 18:53:23.790849 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.790707 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b980862e-40a0-4c8c-8b05-f3c322b397be" containerName="istio-proxy" Jul 10 18:53:23.790849 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.790843 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="258f6865-8c58-4609-85cb-2a8a2212dc2a" containerName="istio-proxy" Jul 10 18:53:23.790957 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.790854 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="258f6865-8c58-4609-85cb-2a8a2212dc2a" containerName="istio-proxy" Jul 10 18:53:23.795602 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.795577 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.799070 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.799050 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-f83faf-openshift-default-dockercfg-bkmjp\"" Jul 10 18:53:23.806477 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.806448 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4"] Jul 10 18:53:23.869844 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.869802 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-podinfo\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.869844 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.869845 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-png99\" (UniqueName: \"kubernetes.io/projected/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-kube-api-access-png99\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.870118 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.869934 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-data\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.870118 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.869977 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-workload-socket\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.870118 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.870042 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istiod-ca-cert\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.870118 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.870109 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-credential-socket\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.870310 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.870134 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-envoy\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.870310 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.870183 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-token\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.870310 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.870237 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-workload-certs\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.971030 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.970994 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istiod-ca-cert\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.971030 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.971043 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-credential-socket\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.971318 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.971058 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-envoy\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.971318 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.971078 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-token\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.971318 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.971116 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-workload-certs\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.971318 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.971230 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-podinfo\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.971318 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.971276 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-png99\" (UniqueName: \"kubernetes.io/projected/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-kube-api-access-png99\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.971607 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.971340 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-data\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.971607 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.971374 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-workload-socket\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.971607 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.971447 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-credential-socket\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.971607 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.971528 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-workload-certs\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.971909 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.971711 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-workload-socket\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.971909 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.971745 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-data\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.971909 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.971847 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istiod-ca-cert\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.973430 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.973400 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-envoy\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.973652 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.973602 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-podinfo\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.980708 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.980678 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-png99\" (UniqueName: \"kubernetes.io/projected/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-kube-api-access-png99\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:23.980708 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:23.980691 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-token\") pod \"e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:24.107360 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:24.107269 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:24.265803 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:53:24.265766 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a4c3aab_359e_444c_9e1e_1f901e3d9f74.slice/crio-ba6f65f5df40b0a73767e9ae3b8523661b888b42d4ad3984e2abe9d93c8bdd5a WatchSource:0}: Error finding container ba6f65f5df40b0a73767e9ae3b8523661b888b42d4ad3984e2abe9d93c8bdd5a: Status 404 returned error can't find the container with id ba6f65f5df40b0a73767e9ae3b8523661b888b42d4ad3984e2abe9d93c8bdd5a Jul 10 18:53:24.268238 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:24.268186 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:53:24.268333 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:24.268278 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:53:24.268333 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:24.268306 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:53:24.275446 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:24.275423 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4"] Jul 10 18:53:24.692047 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:24.692007 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" event={"ID":"1a4c3aab-359e-444c-9e1e-1f901e3d9f74","Type":"ContainerStarted","Data":"39081b8409036e8e532a97a3566d7c1ed4cb0e525a3123b30e7ef5910641cb80"} Jul 10 18:53:24.692275 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:24.692075 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" event={"ID":"1a4c3aab-359e-444c-9e1e-1f901e3d9f74","Type":"ContainerStarted","Data":"ba6f65f5df40b0a73767e9ae3b8523661b888b42d4ad3984e2abe9d93c8bdd5a"} Jul 10 18:53:24.727310 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:24.727168 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" podStartSLOduration=1.727145897 podStartE2EDuration="1.727145897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:53:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:53:24.726399659 +0000 UTC m=+2161.625668768" watchObservedRunningTime="2026-07-10 18:53:24.727145897 +0000 UTC m=+2161.626415004" Jul 10 18:53:25.107709 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:25.107669 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:25.113059 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:25.113022 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:25.697174 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:25.697143 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:25.698110 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:25.698091 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:36.772082 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:36.772046 2571 generic.go:356] "Generic (PLEG): container finished" podID="cccc2f13-7344-4b6f-9788-ef064d1dfde4" containerID="31f82f9f4ca1e4f9108fa3263f814a507d0c3166f7eb9531acd6244184ac08a6" exitCode=0 Jul 10 18:53:36.772650 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:36.772121 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" event={"ID":"cccc2f13-7344-4b6f-9788-ef064d1dfde4","Type":"ContainerDied","Data":"31f82f9f4ca1e4f9108fa3263f814a507d0c3166f7eb9531acd6244184ac08a6"} Jul 10 18:53:36.772772 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:36.772672 2571 scope.go:117] "RemoveContainer" containerID="31f82f9f4ca1e4f9108fa3263f814a507d0c3166f7eb9531acd6244184ac08a6" Jul 10 18:53:36.773902 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:36.773870 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:53:36.774006 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:36.773945 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:53:36.774006 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:36.773983 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:53:37.779428 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:37.779388 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" event={"ID":"cccc2f13-7344-4b6f-9788-ef064d1dfde4","Type":"ContainerStarted","Data":"ab8a8ec2ef23a20a2c95bb9eab1b937d21b6f4bda0766936e27e1abf2e42ebb2"} Jul 10 18:53:38.239483 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:38.239439 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:38.239728 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:38.239585 2571 patch_prober.go:28] interesting pod/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 10 18:53:38.239728 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:38.239653 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" podUID="cccc2f13-7344-4b6f-9788-ef064d1dfde4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 10 18:53:39.239350 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:39.239305 2571 patch_prober.go:28] interesting pod/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 10 18:53:39.239906 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:39.239374 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" podUID="cccc2f13-7344-4b6f-9788-ef064d1dfde4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 10 18:53:40.239737 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:40.239700 2571 patch_prober.go:28] interesting pod/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 10 18:53:40.241077 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:40.241047 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" podUID="cccc2f13-7344-4b6f-9788-ef064d1dfde4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 10 18:53:41.240361 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:41.240317 2571 patch_prober.go:28] interesting pod/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 10 18:53:41.241096 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:41.241055 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" podUID="cccc2f13-7344-4b6f-9788-ef064d1dfde4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 10 18:53:42.243240 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:42.243208 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:42.243762 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:42.243608 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:42.244531 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:42.244508 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:53:44.820148 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:44.820111 2571 generic.go:356] "Generic (PLEG): container finished" podID="1a4c3aab-359e-444c-9e1e-1f901e3d9f74" containerID="39081b8409036e8e532a97a3566d7c1ed4cb0e525a3123b30e7ef5910641cb80" exitCode=0 Jul 10 18:53:44.820728 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:44.820158 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" event={"ID":"1a4c3aab-359e-444c-9e1e-1f901e3d9f74","Type":"ContainerDied","Data":"39081b8409036e8e532a97a3566d7c1ed4cb0e525a3123b30e7ef5910641cb80"} Jul 10 18:53:44.820728 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:44.820568 2571 scope.go:117] "RemoveContainer" containerID="39081b8409036e8e532a97a3566d7c1ed4cb0e525a3123b30e7ef5910641cb80" Jul 10 18:53:44.821802 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:44.821774 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:53:44.821892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:44.821847 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:53:44.821892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:44.821882 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 10 18:53:45.852197 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:45.852162 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" event={"ID":"1a4c3aab-359e-444c-9e1e-1f901e3d9f74","Type":"ContainerStarted","Data":"2a881be66b6366e510d6d03290ed2c3e885ebc98d65f394c1d9c33d0e312f7e1"} Jul 10 18:53:46.108161 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:46.108062 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:46.108335 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:46.108320 2571 patch_prober.go:28] interesting pod/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 10 18:53:46.108391 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:46.108370 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" podUID="1a4c3aab-359e-444c-9e1e-1f901e3d9f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 10 18:53:47.108419 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:47.108377 2571 patch_prober.go:28] interesting pod/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 10 18:53:47.108882 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:47.108442 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" podUID="1a4c3aab-359e-444c-9e1e-1f901e3d9f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 10 18:53:48.107751 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:48.107711 2571 patch_prober.go:28] interesting pod/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 10 18:53:48.107945 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:48.107771 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" podUID="1a4c3aab-359e-444c-9e1e-1f901e3d9f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 10 18:53:49.108305 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:49.108266 2571 patch_prober.go:28] interesting pod/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 10 18:53:49.108750 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:49.108329 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" podUID="1a4c3aab-359e-444c-9e1e-1f901e3d9f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 10 18:53:50.112259 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:50.112225 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:50.112735 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:50.112639 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:53:50.113374 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:53:50.113350 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:56:36.575661 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.575609 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt"] Jul 10 18:56:36.576133 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.576024 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="258f6865-8c58-4609-85cb-2a8a2212dc2a" containerName="istio-proxy" Jul 10 18:56:36.576133 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.576035 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="258f6865-8c58-4609-85cb-2a8a2212dc2a" containerName="istio-proxy" Jul 10 18:56:36.579227 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.579208 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:36.583506 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.583477 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-afdf1e\"/\"openshift-service-ca.crt\"" Jul 10 18:56:36.583721 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.583516 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-afdf1e\"/\"sub-test-model-afdf1e-kserve-self-signed-certs\"" Jul 10 18:56:36.583721 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.583585 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-afdf1e\"/\"kube-root-ca.crt\"" Jul 10 18:56:36.583721 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.583657 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-afdf1e\"/\"default-dockercfg-vqscp\"" Jul 10 18:56:36.588835 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.588810 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt"] Jul 10 18:56:36.639551 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.639513 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-home\") pod \"sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:36.639740 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.639565 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02e3c800-5aee-4d0d-a8aa-428d904dca11-tls-certs\") pod \"sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:36.639740 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.639585 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-model-cache\") pod \"sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:36.639740 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.639721 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-dshm\") pod \"sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:36.639884 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.639772 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrqt7\" (UniqueName: \"kubernetes.io/projected/02e3c800-5aee-4d0d-a8aa-428d904dca11-kube-api-access-qrqt7\") pod \"sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:36.639884 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.639817 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-tmp-dir\") pod \"sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:36.741312 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.741273 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrqt7\" (UniqueName: \"kubernetes.io/projected/02e3c800-5aee-4d0d-a8aa-428d904dca11-kube-api-access-qrqt7\") pod \"sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:36.741312 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.741324 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-tmp-dir\") pod \"sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:36.741586 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.741377 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-home\") pod \"sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:36.741586 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.741408 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02e3c800-5aee-4d0d-a8aa-428d904dca11-tls-certs\") pod \"sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:36.741586 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.741434 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-model-cache\") pod \"sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:36.741586 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.741488 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-dshm\") pod \"sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:36.741907 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.741886 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-home\") pod \"sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:36.742009 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.741925 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-tmp-dir\") pod \"sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:36.742009 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.741948 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-model-cache\") pod \"sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:36.743900 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.743824 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-dshm\") pod \"sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:36.744228 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.744206 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02e3c800-5aee-4d0d-a8aa-428d904dca11-tls-certs\") pod \"sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:36.751212 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.751180 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrqt7\" (UniqueName: \"kubernetes.io/projected/02e3c800-5aee-4d0d-a8aa-428d904dca11-kube-api-access-qrqt7\") pod \"sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:36.889845 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:36.889745 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:37.042029 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:37.042001 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt"] Jul 10 18:56:37.044296 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:56:37.044267 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02e3c800_5aee_4d0d_a8aa_428d904dca11.slice/crio-bb6accfb29a993980ea34c3fe7a6854628cb8f4eed419cdba6033e9c469de25f WatchSource:0}: Error finding container bb6accfb29a993980ea34c3fe7a6854628cb8f4eed419cdba6033e9c469de25f: Status 404 returned error can't find the container with id bb6accfb29a993980ea34c3fe7a6854628cb8f4eed419cdba6033e9c469de25f Jul 10 18:56:37.799187 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:37.799145 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" event={"ID":"02e3c800-5aee-4d0d-a8aa-428d904dca11","Type":"ContainerStarted","Data":"ad2ee9281d258e11520544dc7253a5996f95a0f341cd6921a961d4aec345f715"} Jul 10 18:56:37.799187 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:37.799189 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" event={"ID":"02e3c800-5aee-4d0d-a8aa-428d904dca11","Type":"ContainerStarted","Data":"bb6accfb29a993980ea34c3fe7a6854628cb8f4eed419cdba6033e9c469de25f"} Jul 10 18:56:37.818057 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:37.817988 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" podStartSLOduration=1.817964725 podStartE2EDuration="1.817964725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:56:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:56:37.81690465 +0000 UTC m=+2354.716173760" watchObservedRunningTime="2026-07-10 18:56:37.817964725 +0000 UTC m=+2354.717233835" Jul 10 18:56:46.890719 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:46.890684 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:46.891255 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:46.890735 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:46.903554 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:46.903526 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:56:47.868824 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:56:47.868787 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:57:23.745026 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:23.744995 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r9vz_0cafad48-3998-465d-b4c8-b5d94cac6b3c/ovn-acl-logging/0.log" Jul 10 18:57:23.749432 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:23.749408 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7r9vz_0cafad48-3998-465d-b4c8-b5d94cac6b3c/ovn-acl-logging/0.log" Jul 10 18:57:31.520710 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:31.520675 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt"] Jul 10 18:57:46.541329 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:46.541211 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" podUID="02e3c800-5aee-4d0d-a8aa-428d904dca11" containerName="main" containerID="cri-o://ad2ee9281d258e11520544dc7253a5996f95a0f341cd6921a961d4aec345f715" gracePeriod=45 Jul 10 18:57:46.802910 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:46.802843 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:57:46.927710 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:46.927675 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-tmp-dir\") pod \"02e3c800-5aee-4d0d-a8aa-428d904dca11\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " Jul 10 18:57:46.927909 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:46.927736 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qrqt7\" (UniqueName: \"kubernetes.io/projected/02e3c800-5aee-4d0d-a8aa-428d904dca11-kube-api-access-qrqt7\") pod \"02e3c800-5aee-4d0d-a8aa-428d904dca11\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " Jul 10 18:57:46.927909 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:46.927798 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-home\") pod \"02e3c800-5aee-4d0d-a8aa-428d904dca11\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " Jul 10 18:57:46.927909 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:46.927828 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02e3c800-5aee-4d0d-a8aa-428d904dca11-tls-certs\") pod \"02e3c800-5aee-4d0d-a8aa-428d904dca11\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " Jul 10 18:57:46.927909 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:46.927907 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-model-cache\") pod \"02e3c800-5aee-4d0d-a8aa-428d904dca11\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " Jul 10 18:57:46.928131 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:46.927941 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-dshm\") pod \"02e3c800-5aee-4d0d-a8aa-428d904dca11\" (UID: \"02e3c800-5aee-4d0d-a8aa-428d904dca11\") " Jul 10 18:57:46.928131 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:46.928029 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-home" (OuterVolumeSpecName: "home") pod "02e3c800-5aee-4d0d-a8aa-428d904dca11" (UID: "02e3c800-5aee-4d0d-a8aa-428d904dca11"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:57:46.928240 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:46.928175 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "02e3c800-5aee-4d0d-a8aa-428d904dca11" (UID: "02e3c800-5aee-4d0d-a8aa-428d904dca11"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:57:46.928240 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:46.928208 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-model-cache" (OuterVolumeSpecName: "model-cache") pod "02e3c800-5aee-4d0d-a8aa-428d904dca11" (UID: "02e3c800-5aee-4d0d-a8aa-428d904dca11"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:57:46.928359 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:46.928298 2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-home\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:57:46.928359 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:46.928324 2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-model-cache\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:57:46.928435 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:46.928366 2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-tmp-dir\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:57:46.930535 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:46.930497 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02e3c800-5aee-4d0d-a8aa-428d904dca11-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "02e3c800-5aee-4d0d-a8aa-428d904dca11" (UID: "02e3c800-5aee-4d0d-a8aa-428d904dca11"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:57:46.930535 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:46.930523 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02e3c800-5aee-4d0d-a8aa-428d904dca11-kube-api-access-qrqt7" (OuterVolumeSpecName: "kube-api-access-qrqt7") pod "02e3c800-5aee-4d0d-a8aa-428d904dca11" (UID: "02e3c800-5aee-4d0d-a8aa-428d904dca11"). InnerVolumeSpecName "kube-api-access-qrqt7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:57:46.930827 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:46.930804 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-dshm" (OuterVolumeSpecName: "dshm") pod "02e3c800-5aee-4d0d-a8aa-428d904dca11" (UID: "02e3c800-5aee-4d0d-a8aa-428d904dca11"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:57:47.029140 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:47.029097 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrqt7\" (UniqueName: \"kubernetes.io/projected/02e3c800-5aee-4d0d-a8aa-428d904dca11-kube-api-access-qrqt7\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:57:47.029140 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:47.029133 2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02e3c800-5aee-4d0d-a8aa-428d904dca11-tls-certs\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:57:47.029140 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:47.029143 2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02e3c800-5aee-4d0d-a8aa-428d904dca11-dshm\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:57:47.185760 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:47.185664 2571 generic.go:356] "Generic (PLEG): container finished" podID="02e3c800-5aee-4d0d-a8aa-428d904dca11" containerID="ad2ee9281d258e11520544dc7253a5996f95a0f341cd6921a961d4aec345f715" exitCode=0 Jul 10 18:57:47.185760 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:47.185714 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" Jul 10 18:57:47.185760 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:47.185736 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" event={"ID":"02e3c800-5aee-4d0d-a8aa-428d904dca11","Type":"ContainerDied","Data":"ad2ee9281d258e11520544dc7253a5996f95a0f341cd6921a961d4aec345f715"} Jul 10 18:57:47.186044 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:47.185782 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt" event={"ID":"02e3c800-5aee-4d0d-a8aa-428d904dca11","Type":"ContainerDied","Data":"bb6accfb29a993980ea34c3fe7a6854628cb8f4eed419cdba6033e9c469de25f"} Jul 10 18:57:47.186044 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:47.185803 2571 scope.go:117] "RemoveContainer" containerID="ad2ee9281d258e11520544dc7253a5996f95a0f341cd6921a961d4aec345f715" Jul 10 18:57:47.195775 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:47.195692 2571 scope.go:117] "RemoveContainer" containerID="ad2ee9281d258e11520544dc7253a5996f95a0f341cd6921a961d4aec345f715" Jul 10 18:57:47.196161 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:57:47.196098 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad2ee9281d258e11520544dc7253a5996f95a0f341cd6921a961d4aec345f715\": container with ID starting with ad2ee9281d258e11520544dc7253a5996f95a0f341cd6921a961d4aec345f715 not found: ID does not exist" containerID="ad2ee9281d258e11520544dc7253a5996f95a0f341cd6921a961d4aec345f715" Jul 10 18:57:47.196161 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:47.196132 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad2ee9281d258e11520544dc7253a5996f95a0f341cd6921a961d4aec345f715"} err="failed to get container status \"ad2ee9281d258e11520544dc7253a5996f95a0f341cd6921a961d4aec345f715\": rpc error: code = NotFound desc = could not find container \"ad2ee9281d258e11520544dc7253a5996f95a0f341cd6921a961d4aec345f715\": container with ID starting with ad2ee9281d258e11520544dc7253a5996f95a0f341cd6921a961d4aec345f715 not found: ID does not exist" Jul 10 18:57:47.220032 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:47.219986 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt"] Jul 10 18:57:47.225300 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:47.225260 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-afdf1e/sub-test-model-afdf1e-kserve-7bbcfc864f-j9lzt"] Jul 10 18:57:47.683984 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:57:47.683947 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02e3c800-5aee-4d0d-a8aa-428d904dca11" path="/var/lib/kubelet/pods/02e3c800-5aee-4d0d-a8aa-428d904dca11/volumes" Jul 10 18:58:56.783665 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:56.783630 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7669-139946653554560"] Jul 10 18:58:56.785928 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:56.784024 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02e3c800-5aee-4d0d-a8aa-428d904dca11" containerName="main" Jul 10 18:58:56.785928 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:56.784035 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="02e3c800-5aee-4d0d-a8aa-428d904dca11" containerName="main" Jul 10 18:58:56.785928 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:56.784131 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="02e3c800-5aee-4d0d-a8aa-428d904dca11" containerName="main" Jul 10 18:58:56.787218 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:56.787202 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946653554560" Jul 10 18:58:56.790067 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:56.789985 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-fc7jf\"" Jul 10 18:58:56.791587 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:56.791562 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7669-139946653554560"] Jul 10 18:58:56.885517 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:56.885471 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f57t\" (UniqueName: \"kubernetes.io/projected/2bbfae8b-9152-4ea6-a323-a98c96717093-kube-api-access-2f57t\") pod \"test-curl-7669-139946653554560\" (UID: \"2bbfae8b-9152-4ea6-a323-a98c96717093\") " pod="openshift-ingress/test-curl-7669-139946653554560" Jul 10 18:58:56.986647 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:56.986569 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f57t\" (UniqueName: \"kubernetes.io/projected/2bbfae8b-9152-4ea6-a323-a98c96717093-kube-api-access-2f57t\") pod \"test-curl-7669-139946653554560\" (UID: \"2bbfae8b-9152-4ea6-a323-a98c96717093\") " pod="openshift-ingress/test-curl-7669-139946653554560" Jul 10 18:58:56.999783 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:56.999750 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f57t\" (UniqueName: \"kubernetes.io/projected/2bbfae8b-9152-4ea6-a323-a98c96717093-kube-api-access-2f57t\") pod \"test-curl-7669-139946653554560\" (UID: \"2bbfae8b-9152-4ea6-a323-a98c96717093\") " pod="openshift-ingress/test-curl-7669-139946653554560" Jul 10 18:58:57.098067 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:57.097963 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946653554560" Jul 10 18:58:57.241149 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:57.241125 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7669-139946653554560"] Jul 10 18:58:57.243109 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:58:57.243079 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2bbfae8b_9152_4ea6_a323_a98c96717093.slice/crio-6c913971dfe229bf167b24a2c7692c4b02b3a26fcac53203eb14b88c140e2242 WatchSource:0}: Error finding container 6c913971dfe229bf167b24a2c7692c4b02b3a26fcac53203eb14b88c140e2242: Status 404 returned error can't find the container with id 6c913971dfe229bf167b24a2c7692c4b02b3a26fcac53203eb14b88c140e2242 Jul 10 18:58:57.244379 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:57.244362 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:58:57.581033 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:57.580972 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7669-139946653554560" event={"ID":"2bbfae8b-9152-4ea6-a323-a98c96717093","Type":"ContainerStarted","Data":"6c913971dfe229bf167b24a2c7692c4b02b3a26fcac53203eb14b88c140e2242"} Jul 10 18:58:58.588087 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:58.588048 2571 generic.go:356] "Generic (PLEG): container finished" podID="2bbfae8b-9152-4ea6-a323-a98c96717093" containerID="c698ffba4fb86f23af54f593232a099f7e2776d7bb877c742685652034e32b2e" exitCode=0 Jul 10 18:58:58.588505 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:58.588128 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7669-139946653554560" event={"ID":"2bbfae8b-9152-4ea6-a323-a98c96717093","Type":"ContainerDied","Data":"c698ffba4fb86f23af54f593232a099f7e2776d7bb877c742685652034e32b2e"} Jul 10 18:58:59.746406 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:59.746369 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946653554560" Jul 10 18:58:59.786237 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:59.786211 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7669-139946653554560_2bbfae8b-9152-4ea6-a323-a98c96717093/test-curl-7669-139946653554560/0.log" Jul 10 18:58:59.814162 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:59.814120 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2f57t\" (UniqueName: \"kubernetes.io/projected/2bbfae8b-9152-4ea6-a323-a98c96717093-kube-api-access-2f57t\") pod \"2bbfae8b-9152-4ea6-a323-a98c96717093\" (UID: \"2bbfae8b-9152-4ea6-a323-a98c96717093\") " Jul 10 18:58:59.814411 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:59.814378 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7669-139946653554560"] Jul 10 18:58:59.816546 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:59.816517 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bbfae8b-9152-4ea6-a323-a98c96717093-kube-api-access-2f57t" (OuterVolumeSpecName: "kube-api-access-2f57t") pod "2bbfae8b-9152-4ea6-a323-a98c96717093" (UID: "2bbfae8b-9152-4ea6-a323-a98c96717093"). InnerVolumeSpecName "kube-api-access-2f57t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:58:59.818316 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:59.818295 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7669-139946653554560"] Jul 10 18:58:59.915551 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:58:59.915436 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2f57t\" (UniqueName: \"kubernetes.io/projected/2bbfae8b-9152-4ea6-a323-a98c96717093-kube-api-access-2f57t\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:59:00.600017 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:00.599992 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7669-139946653554560" Jul 10 18:59:00.600017 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:00.600012 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6c913971dfe229bf167b24a2c7692c4b02b3a26fcac53203eb14b88c140e2242" Jul 10 18:59:00.610141 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:59:00.610108 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7669-139946653554560\" is forbidden: User \"system:node:ip-10-0-143-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-143-54.ec2.internal' and this object" podUID="2bbfae8b-9152-4ea6-a323-a98c96717093" pod="openshift-ingress/test-curl-7669-139946653554560" Jul 10 18:59:01.683469 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:01.683434 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bbfae8b-9152-4ea6-a323-a98c96717093" path="/var/lib/kubelet/pods/2bbfae8b-9152-4ea6-a323-a98c96717093/volumes" Jul 10 18:59:16.411208 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:16.411174 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g"] Jul 10 18:59:16.411721 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:16.411435 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" podUID="cccc2f13-7344-4b6f-9788-ef064d1dfde4" containerName="istio-proxy" containerID="cri-o://ab8a8ec2ef23a20a2c95bb9eab1b937d21b6f4bda0766936e27e1abf2e42ebb2" gracePeriod=30 Jul 10 18:59:21.682712 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.682635 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:59:21.719500 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.719454 2571 generic.go:356] "Generic (PLEG): container finished" podID="cccc2f13-7344-4b6f-9788-ef064d1dfde4" containerID="ab8a8ec2ef23a20a2c95bb9eab1b937d21b6f4bda0766936e27e1abf2e42ebb2" exitCode=0 Jul 10 18:59:21.719708 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.719587 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" Jul 10 18:59:21.719708 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.719580 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" event={"ID":"cccc2f13-7344-4b6f-9788-ef064d1dfde4","Type":"ContainerDied","Data":"ab8a8ec2ef23a20a2c95bb9eab1b937d21b6f4bda0766936e27e1abf2e42ebb2"} Jul 10 18:59:21.719708 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.719694 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g" event={"ID":"cccc2f13-7344-4b6f-9788-ef064d1dfde4","Type":"ContainerDied","Data":"6598d626cb2066f6a88668f662624778f96847ab43f10e0671858c5d0b3cbe58"} Jul 10 18:59:21.719830 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.719728 2571 scope.go:117] "RemoveContainer" containerID="ab8a8ec2ef23a20a2c95bb9eab1b937d21b6f4bda0766936e27e1abf2e42ebb2" Jul 10 18:59:21.729973 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.729946 2571 scope.go:117] "RemoveContainer" containerID="31f82f9f4ca1e4f9108fa3263f814a507d0c3166f7eb9531acd6244184ac08a6" Jul 10 18:59:21.739140 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.739118 2571 scope.go:117] "RemoveContainer" containerID="ab8a8ec2ef23a20a2c95bb9eab1b937d21b6f4bda0766936e27e1abf2e42ebb2" Jul 10 18:59:21.739473 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:59:21.739452 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab8a8ec2ef23a20a2c95bb9eab1b937d21b6f4bda0766936e27e1abf2e42ebb2\": container with ID starting with ab8a8ec2ef23a20a2c95bb9eab1b937d21b6f4bda0766936e27e1abf2e42ebb2 not found: ID does not exist" containerID="ab8a8ec2ef23a20a2c95bb9eab1b937d21b6f4bda0766936e27e1abf2e42ebb2" Jul 10 18:59:21.739565 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.739491 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab8a8ec2ef23a20a2c95bb9eab1b937d21b6f4bda0766936e27e1abf2e42ebb2"} err="failed to get container status \"ab8a8ec2ef23a20a2c95bb9eab1b937d21b6f4bda0766936e27e1abf2e42ebb2\": rpc error: code = NotFound desc = could not find container \"ab8a8ec2ef23a20a2c95bb9eab1b937d21b6f4bda0766936e27e1abf2e42ebb2\": container with ID starting with ab8a8ec2ef23a20a2c95bb9eab1b937d21b6f4bda0766936e27e1abf2e42ebb2 not found: ID does not exist" Jul 10 18:59:21.739565 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.739519 2571 scope.go:117] "RemoveContainer" containerID="31f82f9f4ca1e4f9108fa3263f814a507d0c3166f7eb9531acd6244184ac08a6" Jul 10 18:59:21.739865 ip-10-0-143-54 kubenswrapper[2571]: E0710 18:59:21.739839 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31f82f9f4ca1e4f9108fa3263f814a507d0c3166f7eb9531acd6244184ac08a6\": container with ID starting with 31f82f9f4ca1e4f9108fa3263f814a507d0c3166f7eb9531acd6244184ac08a6 not found: ID does not exist" containerID="31f82f9f4ca1e4f9108fa3263f814a507d0c3166f7eb9531acd6244184ac08a6" Jul 10 18:59:21.739912 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.739872 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31f82f9f4ca1e4f9108fa3263f814a507d0c3166f7eb9531acd6244184ac08a6"} err="failed to get container status \"31f82f9f4ca1e4f9108fa3263f814a507d0c3166f7eb9531acd6244184ac08a6\": rpc error: code = NotFound desc = could not find container \"31f82f9f4ca1e4f9108fa3263f814a507d0c3166f7eb9531acd6244184ac08a6\": container with ID starting with 31f82f9f4ca1e4f9108fa3263f814a507d0c3166f7eb9531acd6244184ac08a6 not found: ID does not exist" Jul 10 18:59:21.822002 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.821955 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cswtr\" (UniqueName: \"kubernetes.io/projected/cccc2f13-7344-4b6f-9788-ef064d1dfde4-kube-api-access-cswtr\") pod \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " Jul 10 18:59:21.822002 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.822007 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-token\") pod \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " Jul 10 18:59:21.822280 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.822036 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-envoy\") pod \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " Jul 10 18:59:21.822280 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.822098 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-podinfo\") pod \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " Jul 10 18:59:21.822280 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.822227 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-data\") pod \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " Jul 10 18:59:21.822493 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.822283 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-credential-socket\") pod \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " Jul 10 18:59:21.822493 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.822339 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istiod-ca-cert\") pod \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " Jul 10 18:59:21.822493 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.822377 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-workload-certs\") pod \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " Jul 10 18:59:21.822493 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.822442 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-workload-socket\") pod \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\" (UID: \"cccc2f13-7344-4b6f-9788-ef064d1dfde4\") " Jul 10 18:59:21.823450 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.823215 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cccc2f13-7344-4b6f-9788-ef064d1dfde4" (UID: "cccc2f13-7344-4b6f-9788-ef064d1dfde4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:59:21.823450 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.823250 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cccc2f13-7344-4b6f-9788-ef064d1dfde4" (UID: "cccc2f13-7344-4b6f-9788-ef064d1dfde4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:59:21.823450 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.823375 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cccc2f13-7344-4b6f-9788-ef064d1dfde4" (UID: "cccc2f13-7344-4b6f-9788-ef064d1dfde4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:59:21.823450 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.823423 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cccc2f13-7344-4b6f-9788-ef064d1dfde4" (UID: "cccc2f13-7344-4b6f-9788-ef064d1dfde4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:59:21.823773 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.823455 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-data" (OuterVolumeSpecName: "istio-data") pod "cccc2f13-7344-4b6f-9788-ef064d1dfde4" (UID: "cccc2f13-7344-4b6f-9788-ef064d1dfde4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:59:21.824555 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.824531 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cccc2f13-7344-4b6f-9788-ef064d1dfde4" (UID: "cccc2f13-7344-4b6f-9788-ef064d1dfde4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:59:21.824555 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.824541 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-token" (OuterVolumeSpecName: "istio-token") pod "cccc2f13-7344-4b6f-9788-ef064d1dfde4" (UID: "cccc2f13-7344-4b6f-9788-ef064d1dfde4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:59:21.824776 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.824758 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cccc2f13-7344-4b6f-9788-ef064d1dfde4-kube-api-access-cswtr" (OuterVolumeSpecName: "kube-api-access-cswtr") pod "cccc2f13-7344-4b6f-9788-ef064d1dfde4" (UID: "cccc2f13-7344-4b6f-9788-ef064d1dfde4"). InnerVolumeSpecName "kube-api-access-cswtr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:59:21.824881 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.824858 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cccc2f13-7344-4b6f-9788-ef064d1dfde4" (UID: "cccc2f13-7344-4b6f-9788-ef064d1dfde4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 18:59:21.924284 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.924242 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cswtr\" (UniqueName: \"kubernetes.io/projected/cccc2f13-7344-4b6f-9788-ef064d1dfde4-kube-api-access-cswtr\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:59:21.924284 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.924277 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-token\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:59:21.924284 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.924286 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-envoy\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:59:21.924284 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.924295 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-podinfo\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:59:21.924703 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.924303 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istio-data\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:59:21.924703 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.924311 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-credential-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:59:21.924703 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.924320 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cccc2f13-7344-4b6f-9788-ef064d1dfde4-istiod-ca-cert\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:59:21.924703 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.924328 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-workload-certs\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:59:21.924703 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:21.924336 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cccc2f13-7344-4b6f-9788-ef064d1dfde4-workload-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:59:22.045147 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:22.045107 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g"] Jul 10 18:59:22.050286 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:22.050254 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-afdf1e-openshift-default-58c4cf5d7-mwq5g"] Jul 10 18:59:23.638021 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:23.637985 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4"] Jul 10 18:59:23.638525 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:23.638268 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" podUID="1a4c3aab-359e-444c-9e1e-1f901e3d9f74" containerName="istio-proxy" containerID="cri-o://2a881be66b6366e510d6d03290ed2c3e885ebc98d65f394c1d9c33d0e312f7e1" gracePeriod=30 Jul 10 18:59:23.684349 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:23.684316 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cccc2f13-7344-4b6f-9788-ef064d1dfde4" path="/var/lib/kubelet/pods/cccc2f13-7344-4b6f-9788-ef064d1dfde4/volumes" Jul 10 18:59:24.327940 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:24.327908 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7dd8f4d8b-bzwj7_246fbc8f-4f72-43b9-b237-3836e6abf287/authorino/0.log" Jul 10 18:59:28.761987 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.761941 2571 generic.go:356] "Generic (PLEG): container finished" podID="1a4c3aab-359e-444c-9e1e-1f901e3d9f74" containerID="2a881be66b6366e510d6d03290ed2c3e885ebc98d65f394c1d9c33d0e312f7e1" exitCode=0 Jul 10 18:59:28.762439 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.762005 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" event={"ID":"1a4c3aab-359e-444c-9e1e-1f901e3d9f74","Type":"ContainerDied","Data":"2a881be66b6366e510d6d03290ed2c3e885ebc98d65f394c1d9c33d0e312f7e1"} Jul 10 18:59:28.762439 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.762072 2571 scope.go:117] "RemoveContainer" containerID="39081b8409036e8e532a97a3566d7c1ed4cb0e525a3123b30e7ef5910641cb80" Jul 10 18:59:28.906403 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.906329 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:59:28.991322 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.991285 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-podinfo\") pod \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " Jul 10 18:59:28.991556 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.991346 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-png99\" (UniqueName: \"kubernetes.io/projected/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-kube-api-access-png99\") pod \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " Jul 10 18:59:28.991556 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.991370 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-token\") pod \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " Jul 10 18:59:28.991556 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.991401 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-data\") pod \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " Jul 10 18:59:28.991556 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.991424 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istiod-ca-cert\") pod \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " Jul 10 18:59:28.991556 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.991449 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-workload-socket\") pod \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " Jul 10 18:59:28.991556 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.991470 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-credential-socket\") pod \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " Jul 10 18:59:28.991556 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.991518 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-envoy\") pod \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " Jul 10 18:59:28.991943 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.991564 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-workload-certs\") pod \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\" (UID: \"1a4c3aab-359e-444c-9e1e-1f901e3d9f74\") " Jul 10 18:59:28.991943 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.991812 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1a4c3aab-359e-444c-9e1e-1f901e3d9f74" (UID: "1a4c3aab-359e-444c-9e1e-1f901e3d9f74"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:59:28.991943 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.991840 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1a4c3aab-359e-444c-9e1e-1f901e3d9f74" (UID: "1a4c3aab-359e-444c-9e1e-1f901e3d9f74"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:59:28.991943 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.991889 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1a4c3aab-359e-444c-9e1e-1f901e3d9f74" (UID: "1a4c3aab-359e-444c-9e1e-1f901e3d9f74"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:59:28.992148 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.992069 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1a4c3aab-359e-444c-9e1e-1f901e3d9f74" (UID: "1a4c3aab-359e-444c-9e1e-1f901e3d9f74"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:59:28.992148 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.992098 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istiod-ca-cert\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:59:28.992148 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.992121 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-workload-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:59:28.992148 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.992134 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-credential-socket\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:59:28.992773 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.992745 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-data" (OuterVolumeSpecName: "istio-data") pod "1a4c3aab-359e-444c-9e1e-1f901e3d9f74" (UID: "1a4c3aab-359e-444c-9e1e-1f901e3d9f74"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:59:28.993741 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.993714 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-token" (OuterVolumeSpecName: "istio-token") pod "1a4c3aab-359e-444c-9e1e-1f901e3d9f74" (UID: "1a4c3aab-359e-444c-9e1e-1f901e3d9f74"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:59:28.993866 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.993738 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1a4c3aab-359e-444c-9e1e-1f901e3d9f74" (UID: "1a4c3aab-359e-444c-9e1e-1f901e3d9f74"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 18:59:28.993866 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.993805 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-kube-api-access-png99" (OuterVolumeSpecName: "kube-api-access-png99") pod "1a4c3aab-359e-444c-9e1e-1f901e3d9f74" (UID: "1a4c3aab-359e-444c-9e1e-1f901e3d9f74"). InnerVolumeSpecName "kube-api-access-png99". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:59:28.994211 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:28.994196 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1a4c3aab-359e-444c-9e1e-1f901e3d9f74" (UID: "1a4c3aab-359e-444c-9e1e-1f901e3d9f74"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 18:59:29.093504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:29.093465 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-podinfo\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:59:29.093504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:29.093498 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-png99\" (UniqueName: \"kubernetes.io/projected/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-kube-api-access-png99\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:59:29.093504 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:29.093511 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-token\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:59:29.093807 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:29.093522 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-data\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:59:29.093807 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:29.093533 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-istio-envoy\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:59:29.093807 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:29.093544 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a4c3aab-359e-444c-9e1e-1f901e3d9f74-workload-certs\") on node \"ip-10-0-143-54.ec2.internal\" DevicePath \"\"" Jul 10 18:59:29.402278 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:29.402243 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-m89t9_147ffda5-ecb7-4e1e-adf7-bb36bdbe42ee/manager/0.log" Jul 10 18:59:29.536892 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:29.536862 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6645c_1c5eed36-9d1c-487a-98f7-a01d4d540b8c/manager/0.log" Jul 10 18:59:29.769021 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:29.768988 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" Jul 10 18:59:29.769021 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:29.769014 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4" event={"ID":"1a4c3aab-359e-444c-9e1e-1f901e3d9f74","Type":"ContainerDied","Data":"ba6f65f5df40b0a73767e9ae3b8523661b888b42d4ad3984e2abe9d93c8bdd5a"} Jul 10 18:59:29.769573 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:29.769067 2571 scope.go:117] "RemoveContainer" containerID="2a881be66b6366e510d6d03290ed2c3e885ebc98d65f394c1d9c33d0e312f7e1" Jul 10 18:59:29.792359 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:29.792325 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4"] Jul 10 18:59:29.797078 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:29.797048 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-f83faf-openshift-default-7f9744676c-qwjj4"] Jul 10 18:59:29.798182 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:29.798163 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-mc5v5_7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d/server/0.log" Jul 10 18:59:29.920692 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:29.920657 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-j46xf_da45c16f-fdef-4fd6-b301-bf1a78e557d7/manager/0.log" Jul 10 18:59:30.038948 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:30.038906 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d75465dc8-27qb2_254586fb-ea8a-4085-97da-1a3c5c95454d/copy-manifests/0.log" Jul 10 18:59:30.047113 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:30.047065 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d75465dc8-27qb2_254586fb-ea8a-4085-97da-1a3c5c95454d/manager/0.log" Jul 10 18:59:31.605088 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:31.605057 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4_a9a4c970-a93f-48fb-82ac-6ecc1663d4b7/pull/0.log" Jul 10 18:59:31.621144 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:31.621118 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4_a9a4c970-a93f-48fb-82ac-6ecc1663d4b7/extract/0.log" Jul 10 18:59:31.635120 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:31.635096 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4_a9a4c970-a93f-48fb-82ac-6ecc1663d4b7/util/0.log" Jul 10 18:59:31.689558 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:31.689526 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a4c3aab-359e-444c-9e1e-1f901e3d9f74" path="/var/lib/kubelet/pods/1a4c3aab-359e-444c-9e1e-1f901e3d9f74/volumes" Jul 10 18:59:31.763380 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:31.763353 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6_7064372d-282f-48d8-a557-e4a5111bca35/util/0.log" Jul 10 18:59:31.772206 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:31.772180 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6_7064372d-282f-48d8-a557-e4a5111bca35/pull/0.log" Jul 10 18:59:31.783561 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:31.783532 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6_7064372d-282f-48d8-a557-e4a5111bca35/extract/0.log" Jul 10 18:59:31.900906 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:31.900822 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf_3f63cc50-f78f-4814-b47d-93b0efefafa1/util/0.log" Jul 10 18:59:31.909082 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:31.909058 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf_3f63cc50-f78f-4814-b47d-93b0efefafa1/pull/0.log" Jul 10 18:59:31.917474 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:31.917439 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf_3f63cc50-f78f-4814-b47d-93b0efefafa1/extract/0.log" Jul 10 18:59:32.034568 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:32.034539 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4_70818f90-e5b1-4107-b55a-8e7a53b96814/util/0.log" Jul 10 18:59:32.042626 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:32.042593 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4_70818f90-e5b1-4107-b55a-8e7a53b96814/pull/0.log" Jul 10 18:59:32.053005 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:32.052983 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4_70818f90-e5b1-4107-b55a-8e7a53b96814/extract/0.log" Jul 10 18:59:32.173429 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:32.173337 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7dd8f4d8b-bzwj7_246fbc8f-4f72-43b9-b237-3836e6abf287/authorino/0.log" Jul 10 18:59:32.296940 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:32.296908 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-lkjnp_f2102ee6-215c-4455-9c64-48d52a6842da/manager/0.log" Jul 10 18:59:32.412831 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:32.412800 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qw6ph_3fac146c-8090-4f9c-bcc4-ca041ffbe7d3/manager/0.log" Jul 10 18:59:32.524882 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:32.524850 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2l5kn_a170d9d2-c194-4b40-af59-80030fdf258f/kuadrant-console-plugin/0.log" Jul 10 18:59:32.642379 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:32.642341 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-kqcg5_e1f1a3b1-0c3a-4ea5-8370-70a73e7904f7/registry-server/0.log" Jul 10 18:59:32.764739 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:32.764711 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-xh2np_164d1efe-3fe0-4f7d-9fd6-1031e530a9d8/manager/0.log" Jul 10 18:59:33.043908 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:33.043878 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-zn4xq_c1fc810b-3327-4e37-b42a-bb3963bafc4d/manager/0.log" Jul 10 18:59:33.400073 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:33.399993 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678hw6hw_82c47cdc-e129-4ad8-92af-8744507f1872/istio-proxy/0.log" Jul 10 18:59:33.750145 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:33.750120 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79bb7c45b8-ss6xx_9764f143-44c0-4790-8b60-530bd022797f/kube-auth-proxy/0.log" Jul 10 18:59:33.875161 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:33.875130 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-msnmz_31a095f9-f006-476e-b42b-d7682d0f463b/istio-proxy/1.log" Jul 10 18:59:34.736053 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:34.736023 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-wxfq5_bcb9bf03-4584-445d-a810-028464c7b9fd/main/0.log" Jul 10 18:59:35.086645 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:35.086591 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-jhxbs_6fa2adcd-bba5-4c01-91b3-7ecef99b5647/main/0.log" Jul 10 18:59:39.197346 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.197314 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9xwdk/must-gather-wt5fz"] Jul 10 18:59:39.197764 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.197710 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cccc2f13-7344-4b6f-9788-ef064d1dfde4" containerName="istio-proxy" Jul 10 18:59:39.197764 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.197721 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="cccc2f13-7344-4b6f-9788-ef064d1dfde4" containerName="istio-proxy" Jul 10 18:59:39.197764 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.197730 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a4c3aab-359e-444c-9e1e-1f901e3d9f74" containerName="istio-proxy" Jul 10 18:59:39.197764 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.197736 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a4c3aab-359e-444c-9e1e-1f901e3d9f74" containerName="istio-proxy" Jul 10 18:59:39.197764 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.197744 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bbfae8b-9152-4ea6-a323-a98c96717093" containerName="test-curl-7669-139946653554560" Jul 10 18:59:39.197764 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.197750 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bbfae8b-9152-4ea6-a323-a98c96717093" containerName="test-curl-7669-139946653554560" Jul 10 18:59:39.197764 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.197761 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cccc2f13-7344-4b6f-9788-ef064d1dfde4" containerName="istio-proxy" Jul 10 18:59:39.197764 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.197768 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="cccc2f13-7344-4b6f-9788-ef064d1dfde4" containerName="istio-proxy" Jul 10 18:59:39.198029 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.197783 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a4c3aab-359e-444c-9e1e-1f901e3d9f74" containerName="istio-proxy" Jul 10 18:59:39.198029 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.197788 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a4c3aab-359e-444c-9e1e-1f901e3d9f74" containerName="istio-proxy" Jul 10 18:59:39.198029 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.197840 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="cccc2f13-7344-4b6f-9788-ef064d1dfde4" containerName="istio-proxy" Jul 10 18:59:39.198029 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.197849 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a4c3aab-359e-444c-9e1e-1f901e3d9f74" containerName="istio-proxy" Jul 10 18:59:39.198029 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.197854 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a4c3aab-359e-444c-9e1e-1f901e3d9f74" containerName="istio-proxy" Jul 10 18:59:39.198029 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.197861 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bbfae8b-9152-4ea6-a323-a98c96717093" containerName="test-curl-7669-139946653554560" Jul 10 18:59:39.198029 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.197967 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="cccc2f13-7344-4b6f-9788-ef064d1dfde4" containerName="istio-proxy" Jul 10 18:59:39.201282 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.201263 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9xwdk/must-gather-wt5fz" Jul 10 18:59:39.204156 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.204124 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9xwdk\"/\"openshift-service-ca.crt\"" Jul 10 18:59:39.204343 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.204169 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9xwdk\"/\"kube-root-ca.crt\"" Jul 10 18:59:39.204343 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.204192 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9xwdk\"/\"default-dockercfg-rjp4f\"" Jul 10 18:59:39.211205 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.211179 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9xwdk/must-gather-wt5fz"] Jul 10 18:59:39.274072 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.274034 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a031e547-b106-4f29-a4a9-804660d33aeb-must-gather-output\") pod \"must-gather-wt5fz\" (UID: \"a031e547-b106-4f29-a4a9-804660d33aeb\") " pod="openshift-must-gather-9xwdk/must-gather-wt5fz" Jul 10 18:59:39.274072 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.274074 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98p9l\" (UniqueName: \"kubernetes.io/projected/a031e547-b106-4f29-a4a9-804660d33aeb-kube-api-access-98p9l\") pod \"must-gather-wt5fz\" (UID: \"a031e547-b106-4f29-a4a9-804660d33aeb\") " pod="openshift-must-gather-9xwdk/must-gather-wt5fz" Jul 10 18:59:39.374744 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.374693 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a031e547-b106-4f29-a4a9-804660d33aeb-must-gather-output\") pod \"must-gather-wt5fz\" (UID: \"a031e547-b106-4f29-a4a9-804660d33aeb\") " pod="openshift-must-gather-9xwdk/must-gather-wt5fz" Jul 10 18:59:39.374744 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.374742 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98p9l\" (UniqueName: \"kubernetes.io/projected/a031e547-b106-4f29-a4a9-804660d33aeb-kube-api-access-98p9l\") pod \"must-gather-wt5fz\" (UID: \"a031e547-b106-4f29-a4a9-804660d33aeb\") " pod="openshift-must-gather-9xwdk/must-gather-wt5fz" Jul 10 18:59:39.375126 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.375104 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a031e547-b106-4f29-a4a9-804660d33aeb-must-gather-output\") pod \"must-gather-wt5fz\" (UID: \"a031e547-b106-4f29-a4a9-804660d33aeb\") " pod="openshift-must-gather-9xwdk/must-gather-wt5fz" Jul 10 18:59:39.383998 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.383959 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98p9l\" (UniqueName: \"kubernetes.io/projected/a031e547-b106-4f29-a4a9-804660d33aeb-kube-api-access-98p9l\") pod \"must-gather-wt5fz\" (UID: \"a031e547-b106-4f29-a4a9-804660d33aeb\") " pod="openshift-must-gather-9xwdk/must-gather-wt5fz" Jul 10 18:59:39.511234 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.511190 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9xwdk/must-gather-wt5fz" Jul 10 18:59:39.658118 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.658087 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9xwdk/must-gather-wt5fz"] Jul 10 18:59:39.659207 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:59:39.659174 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda031e547_b106_4f29_a4a9_804660d33aeb.slice/crio-138bccb4d3b17f75144f77d7fd5957a79652dd1f76caebb3402a52ea809b0628 WatchSource:0}: Error finding container 138bccb4d3b17f75144f77d7fd5957a79652dd1f76caebb3402a52ea809b0628: Status 404 returned error can't find the container with id 138bccb4d3b17f75144f77d7fd5957a79652dd1f76caebb3402a52ea809b0628 Jul 10 18:59:39.821441 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:39.821351 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9xwdk/must-gather-wt5fz" event={"ID":"a031e547-b106-4f29-a4a9-804660d33aeb","Type":"ContainerStarted","Data":"138bccb4d3b17f75144f77d7fd5957a79652dd1f76caebb3402a52ea809b0628"} Jul 10 18:59:40.832460 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:40.832424 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9xwdk/must-gather-wt5fz" event={"ID":"a031e547-b106-4f29-a4a9-804660d33aeb","Type":"ContainerStarted","Data":"0b5043565769ca9e810bfc07b638ecffd26b71790d86d0631f9c7fb013678d03"} Jul 10 18:59:40.832894 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:40.832472 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9xwdk/must-gather-wt5fz" event={"ID":"a031e547-b106-4f29-a4a9-804660d33aeb","Type":"ContainerStarted","Data":"ddaeb0a8c5348b92bc652318d970639f009cca2147c22225c57f13df67eda49d"} Jul 10 18:59:40.852923 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:40.852857 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9xwdk/must-gather-wt5fz" podStartSLOduration=1.059450305 podStartE2EDuration="1.852833969s" totalImagesPullingTime="793.383664ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:59:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:59:40.852384083 +0000 UTC m=+2537.751653192" watchObservedRunningTime="2026-07-10 18:59:40.852833969 +0000 UTC m=+2537.752103078" Jul 10 18:59:42.193059 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:42.193021 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9rqql_91c02392-14bc-403d-ab9f-131ed9213d28/global-pull-secret-syncer/0.log" Jul 10 18:59:42.345477 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:42.345447 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hn6g2_3fbdae78-1e8a-4b53-aff0-0f807312bd7e/konnectivity-agent/0.log" Jul 10 18:59:42.460776 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:42.460699 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-54.ec2.internal_33a81504b2c888f51c5efff3d00523ac/haproxy/0.log" Jul 10 18:59:47.460260 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:47.460222 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4_a9a4c970-a93f-48fb-82ac-6ecc1663d4b7/extract/0.log" Jul 10 18:59:47.487903 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:47.487869 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4_a9a4c970-a93f-48fb-82ac-6ecc1663d4b7/util/0.log" Jul 10 18:59:47.520222 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:47.520188 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n22t4_a9a4c970-a93f-48fb-82ac-6ecc1663d4b7/pull/0.log" Jul 10 18:59:47.584069 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:47.584031 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6_7064372d-282f-48d8-a557-e4a5111bca35/extract/0.log" Jul 10 18:59:47.624870 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:47.624833 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6_7064372d-282f-48d8-a557-e4a5111bca35/util/0.log" Jul 10 18:59:47.653306 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:47.653251 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02n5n6_7064372d-282f-48d8-a557-e4a5111bca35/pull/0.log" Jul 10 18:59:47.691636 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:47.691571 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf_3f63cc50-f78f-4814-b47d-93b0efefafa1/extract/0.log" Jul 10 18:59:47.727958 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:47.727866 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf_3f63cc50-f78f-4814-b47d-93b0efefafa1/util/0.log" Jul 10 18:59:47.755231 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:47.755130 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jmsxf_3f63cc50-f78f-4814-b47d-93b0efefafa1/pull/0.log" Jul 10 18:59:47.794488 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:47.794456 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4_70818f90-e5b1-4107-b55a-8e7a53b96814/extract/0.log" Jul 10 18:59:47.855654 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:47.855599 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4_70818f90-e5b1-4107-b55a-8e7a53b96814/util/0.log" Jul 10 18:59:47.919865 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:47.919810 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1b7fv4_70818f90-e5b1-4107-b55a-8e7a53b96814/pull/0.log" Jul 10 18:59:48.376272 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:48.376234 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7dd8f4d8b-bzwj7_246fbc8f-4f72-43b9-b237-3836e6abf287/authorino/0.log" Jul 10 18:59:48.414644 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:48.414572 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-lkjnp_f2102ee6-215c-4455-9c64-48d52a6842da/manager/0.log" Jul 10 18:59:48.451447 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:48.451408 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qw6ph_3fac146c-8090-4f9c-bcc4-ca041ffbe7d3/manager/0.log" Jul 10 18:59:48.485901 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:48.485866 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2l5kn_a170d9d2-c194-4b40-af59-80030fdf258f/kuadrant-console-plugin/0.log" Jul 10 18:59:48.534717 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:48.534663 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-kqcg5_e1f1a3b1-0c3a-4ea5-8370-70a73e7904f7/registry-server/0.log" Jul 10 18:59:48.727682 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:48.727566 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-xh2np_164d1efe-3fe0-4f7d-9fd6-1031e530a9d8/manager/0.log" Jul 10 18:59:48.843340 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:48.843301 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-zn4xq_c1fc810b-3327-4e37-b42a-bb3963bafc4d/manager/0.log" Jul 10 18:59:50.844799 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:50.844770 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2rz6f_8b997f6d-4391-4ecb-bb62-125e3a906d63/node-exporter/0.log" Jul 10 18:59:50.879765 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:50.879731 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2rz6f_8b997f6d-4391-4ecb-bb62-125e3a906d63/kube-rbac-proxy/0.log" Jul 10 18:59:50.904896 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:50.904868 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2rz6f_8b997f6d-4391-4ecb-bb62-125e3a906d63/init-textfile/0.log" Jul 10 18:59:52.138338 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.138299 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6"] Jul 10 18:59:52.144330 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.144296 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" Jul 10 18:59:52.156759 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.156725 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6"] Jul 10 18:59:52.212640 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.211880 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6a315cc-9d1e-4431-b4e2-11b3ea389187-sys\") pod \"perf-node-gather-daemonset-x8lt6\" (UID: \"a6a315cc-9d1e-4431-b4e2-11b3ea389187\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" Jul 10 18:59:52.212640 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.211943 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a6a315cc-9d1e-4431-b4e2-11b3ea389187-podres\") pod \"perf-node-gather-daemonset-x8lt6\" (UID: \"a6a315cc-9d1e-4431-b4e2-11b3ea389187\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" Jul 10 18:59:52.212640 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.212000 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n52gl\" (UniqueName: \"kubernetes.io/projected/a6a315cc-9d1e-4431-b4e2-11b3ea389187-kube-api-access-n52gl\") pod \"perf-node-gather-daemonset-x8lt6\" (UID: \"a6a315cc-9d1e-4431-b4e2-11b3ea389187\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" Jul 10 18:59:52.212640 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.212041 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a6a315cc-9d1e-4431-b4e2-11b3ea389187-lib-modules\") pod \"perf-node-gather-daemonset-x8lt6\" (UID: \"a6a315cc-9d1e-4431-b4e2-11b3ea389187\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" Jul 10 18:59:52.212640 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.212091 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a6a315cc-9d1e-4431-b4e2-11b3ea389187-proc\") pod \"perf-node-gather-daemonset-x8lt6\" (UID: \"a6a315cc-9d1e-4431-b4e2-11b3ea389187\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" Jul 10 18:59:52.313316 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.313268 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n52gl\" (UniqueName: \"kubernetes.io/projected/a6a315cc-9d1e-4431-b4e2-11b3ea389187-kube-api-access-n52gl\") pod \"perf-node-gather-daemonset-x8lt6\" (UID: \"a6a315cc-9d1e-4431-b4e2-11b3ea389187\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" Jul 10 18:59:52.313539 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.313337 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a6a315cc-9d1e-4431-b4e2-11b3ea389187-lib-modules\") pod \"perf-node-gather-daemonset-x8lt6\" (UID: \"a6a315cc-9d1e-4431-b4e2-11b3ea389187\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" Jul 10 18:59:52.313539 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.313383 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a6a315cc-9d1e-4431-b4e2-11b3ea389187-proc\") pod \"perf-node-gather-daemonset-x8lt6\" (UID: \"a6a315cc-9d1e-4431-b4e2-11b3ea389187\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" Jul 10 18:59:52.313539 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.313471 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6a315cc-9d1e-4431-b4e2-11b3ea389187-sys\") pod \"perf-node-gather-daemonset-x8lt6\" (UID: \"a6a315cc-9d1e-4431-b4e2-11b3ea389187\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" Jul 10 18:59:52.313539 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.313501 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a6a315cc-9d1e-4431-b4e2-11b3ea389187-podres\") pod \"perf-node-gather-daemonset-x8lt6\" (UID: \"a6a315cc-9d1e-4431-b4e2-11b3ea389187\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" Jul 10 18:59:52.313794 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.313538 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a6a315cc-9d1e-4431-b4e2-11b3ea389187-lib-modules\") pod \"perf-node-gather-daemonset-x8lt6\" (UID: \"a6a315cc-9d1e-4431-b4e2-11b3ea389187\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" Jul 10 18:59:52.313794 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.313623 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6a315cc-9d1e-4431-b4e2-11b3ea389187-sys\") pod \"perf-node-gather-daemonset-x8lt6\" (UID: \"a6a315cc-9d1e-4431-b4e2-11b3ea389187\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" Jul 10 18:59:52.313794 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.313659 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a6a315cc-9d1e-4431-b4e2-11b3ea389187-podres\") pod \"perf-node-gather-daemonset-x8lt6\" (UID: \"a6a315cc-9d1e-4431-b4e2-11b3ea389187\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" Jul 10 18:59:52.313794 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.313710 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a6a315cc-9d1e-4431-b4e2-11b3ea389187-proc\") pod \"perf-node-gather-daemonset-x8lt6\" (UID: \"a6a315cc-9d1e-4431-b4e2-11b3ea389187\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" Jul 10 18:59:52.322577 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.322541 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n52gl\" (UniqueName: \"kubernetes.io/projected/a6a315cc-9d1e-4431-b4e2-11b3ea389187-kube-api-access-n52gl\") pod \"perf-node-gather-daemonset-x8lt6\" (UID: \"a6a315cc-9d1e-4431-b4e2-11b3ea389187\") " pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" Jul 10 18:59:52.458717 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.458634 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" Jul 10 18:59:52.665408 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.665365 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6"] Jul 10 18:59:52.668962 ip-10-0-143-54 kubenswrapper[2571]: W0710 18:59:52.668930 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda6a315cc_9d1e_4431_b4e2_11b3ea389187.slice/crio-f642ba7b6a7d26451ed68c742fc276953eae2d6fad731131d65a77ba6123bfc5 WatchSource:0}: Error finding container f642ba7b6a7d26451ed68c742fc276953eae2d6fad731131d65a77ba6123bfc5: Status 404 returned error can't find the container with id f642ba7b6a7d26451ed68c742fc276953eae2d6fad731131d65a77ba6123bfc5 Jul 10 18:59:52.933093 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.933046 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" event={"ID":"a6a315cc-9d1e-4431-b4e2-11b3ea389187","Type":"ContainerStarted","Data":"2e482f7ccf0482ed82586340d0866163b453293a6b2c9a7d16995cfe4f010369"} Jul 10 18:59:52.933308 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.933120 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" event={"ID":"a6a315cc-9d1e-4431-b4e2-11b3ea389187","Type":"ContainerStarted","Data":"f642ba7b6a7d26451ed68c742fc276953eae2d6fad731131d65a77ba6123bfc5"} Jul 10 18:59:52.933308 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.933177 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" Jul 10 18:59:52.955192 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:52.955108 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" podStartSLOduration=0.955069187 podStartE2EDuration="955.069187ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:59:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:59:52.952922201 +0000 UTC m=+2549.852191335" watchObservedRunningTime="2026-07-10 18:59:52.955069187 +0000 UTC m=+2549.854338297" Jul 10 18:59:55.884515 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:55.884487 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xksd4_2f17d7a9-1710-4246-9196-6c85ce329679/dns/0.log" Jul 10 18:59:55.912526 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:55.912475 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xksd4_2f17d7a9-1710-4246-9196-6c85ce329679/kube-rbac-proxy/0.log" Jul 10 18:59:55.971042 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:55.971010 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kt69c_05e2da1a-b49d-47e5-85e8-2ce1d30d3aae/dns-node-resolver/0.log" Jul 10 18:59:56.588082 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:56.588040 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6jpkg_a3929aa3-2abd-489c-b7fd-c80e982db8e8/node-ca/0.log" Jul 10 18:59:57.695999 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:57.695972 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678hw6hw_82c47cdc-e129-4ad8-92af-8744507f1872/istio-proxy/0.log" Jul 10 18:59:57.875761 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:57.875727 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79bb7c45b8-ss6xx_9764f143-44c0-4790-8b60-530bd022797f/kube-auth-proxy/0.log" Jul 10 18:59:57.917157 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:57.917117 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-msnmz_31a095f9-f006-476e-b42b-d7682d0f463b/istio-proxy/0.log" Jul 10 18:59:57.960098 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:57.960011 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-msnmz_31a095f9-f006-476e-b42b-d7682d0f463b/istio-proxy/1.log" Jul 10 18:59:58.787787 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:58.787760 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cwtqn_95b3a951-0550-4589-afc1-3255ff58e1ec/serve-healthcheck-canary/0.log" Jul 10 18:59:58.951250 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:58.951220 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9xwdk/perf-node-gather-daemonset-x8lt6" Jul 10 18:59:59.439111 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:59.439084 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5hhgq_92eb18e4-693d-4dd4-8948-501fbc8571eb/kube-rbac-proxy/0.log" Jul 10 18:59:59.467419 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:59.467396 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5hhgq_92eb18e4-693d-4dd4-8948-501fbc8571eb/exporter/0.log" Jul 10 18:59:59.495998 ip-10-0-143-54 kubenswrapper[2571]: I0710 18:59:59.495967 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5hhgq_92eb18e4-693d-4dd4-8948-501fbc8571eb/extractor/0.log" Jul 10 19:00:02.118149 ip-10-0-143-54 kubenswrapper[2571]: I0710 19:00:02.118113 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-m89t9_147ffda5-ecb7-4e1e-adf7-bb36bdbe42ee/manager/0.log" Jul 10 19:00:02.312520 ip-10-0-143-54 kubenswrapper[2571]: I0710 19:00:02.312486 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6645c_1c5eed36-9d1c-487a-98f7-a01d4d540b8c/manager/0.log" Jul 10 19:00:02.440905 ip-10-0-143-54 kubenswrapper[2571]: I0710 19:00:02.440800 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-mc5v5_7791dd5d-23fb-4d0f-98ae-e30e0dfcb94d/server/0.log" Jul 10 19:00:02.524741 ip-10-0-143-54 kubenswrapper[2571]: I0710 19:00:02.524698 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-j46xf_da45c16f-fdef-4fd6-b301-bf1a78e557d7/manager/0.log" Jul 10 19:00:02.554227 ip-10-0-143-54 kubenswrapper[2571]: I0710 19:00:02.554186 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d75465dc8-27qb2_254586fb-ea8a-4085-97da-1a3c5c95454d/manager/0.log" Jul 10 19:00:02.581225 ip-10-0-143-54 kubenswrapper[2571]: I0710 19:00:02.581199 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d75465dc8-27qb2_254586fb-ea8a-4085-97da-1a3c5c95454d/copy-manifests/0.log" Jul 10 19:00:03.306145 ip-10-0-143-54 kubenswrapper[2571]: I0710 19:00:03.306115 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678hw6hw_82c47cdc-e129-4ad8-92af-8744507f1872/istio-proxy/0.log" Jul 10 19:00:03.447445 ip-10-0-143-54 kubenswrapper[2571]: I0710 19:00:03.447387 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79bb7c45b8-ss6xx_9764f143-44c0-4790-8b60-530bd022797f/kube-auth-proxy/0.log" Jul 10 19:00:03.469912 ip-10-0-143-54 kubenswrapper[2571]: I0710 19:00:03.469882 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-msnmz_31a095f9-f006-476e-b42b-d7682d0f463b/istio-proxy/0.log" Jul 10 19:00:03.509177 ip-10-0-143-54 kubenswrapper[2571]: I0710 19:00:03.509144 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-msnmz_31a095f9-f006-476e-b42b-d7682d0f463b/istio-proxy/1.log" Jul 10 19:00:04.810664 ip-10-0-143-54 kubenswrapper[2571]: I0710 19:00:04.810629 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678hw6hw_82c47cdc-e129-4ad8-92af-8744507f1872/istio-proxy/0.log" Jul 10 19:00:04.948531 ip-10-0-143-54 kubenswrapper[2571]: I0710 19:00:04.948501 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79bb7c45b8-ss6xx_9764f143-44c0-4790-8b60-530bd022797f/kube-auth-proxy/0.log" Jul 10 19:00:04.971169 ip-10-0-143-54 kubenswrapper[2571]: I0710 19:00:04.971136 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-msnmz_31a095f9-f006-476e-b42b-d7682d0f463b/istio-proxy/0.log" Jul 10 19:00:05.012714 ip-10-0-143-54 kubenswrapper[2571]: I0710 19:00:05.012678 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-msnmz_31a095f9-f006-476e-b42b-d7682d0f463b/istio-proxy/1.log" Jul 10 19:00:09.089365 ip-10-0-143-54 kubenswrapper[2571]: I0710 19:00:09.089335 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-v6xpt_28465aca-96df-43d6-8c90-42a88fb716ca/migrator/0.log" Jul 10 19:00:09.114240 ip-10-0-143-54 kubenswrapper[2571]: I0710 19:00:09.114213 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-v6xpt_28465aca-96df-43d6-8c90-42a88fb716ca/graceful-termination/0.log"