Jul 14 13:25:05.706920 ip-10-0-142-94 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 14 13:25:05.706929 ip-10-0-142-94 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 14 13:25:05.706936 ip-10-0-142-94 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 14 13:25:05.707159 ip-10-0-142-94 systemd[1]: Failed to start Kubernetes Kubelet. Jul 14 13:25:15.934695 ip-10-0-142-94 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 14 13:25:15.934719 ip-10-0-142-94 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot e21a2f6d70e14508ae282069f1e594a9 -- Jul 14 13:27:16.315425 ip-10-0-142-94 systemd[1]: Starting Kubernetes Kubelet... Jul 14 13:27:17.040328 ip-10-0-142-94 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 13:27:17.040328 ip-10-0-142-94 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 14 13:27:17.040328 ip-10-0-142-94 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 13:27:17.040328 ip-10-0-142-94 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 14 13:27:17.040328 ip-10-0-142-94 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 13:27:17.040328 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.959831 2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 14 13:27:17.040328 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965661 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 13:27:17.040328 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965676 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 13:27:17.040328 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965680 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 13:27:17.040328 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965683 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 13:27:17.040328 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965686 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 13:27:17.040328 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965689 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 13:27:17.040328 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965692 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 13:27:17.040328 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965696 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 13:27:17.042094 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965699 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 13:27:17.042094 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965702 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 13:27:17.042094 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965705 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 13:27:17.042094 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965708 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 13:27:17.042094 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965710 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 13:27:17.042094 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965714 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 13:27:17.042094 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965718 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 13:27:17.042094 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965721 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 13:27:17.042094 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965723 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 13:27:17.042094 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965726 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 13:27:17.042094 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965729 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 13:27:17.042094 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965732 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 13:27:17.042094 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965734 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 13:27:17.042094 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965737 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 13:27:17.042094 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965740 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 13:27:17.042094 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965743 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 13:27:17.042094 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965746 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 13:27:17.042094 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965751 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 13:27:17.042094 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965756 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 13:27:17.042094 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965759 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 13:27:17.042949 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965761 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 13:27:17.042949 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965765 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 13:27:17.042949 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965767 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 13:27:17.042949 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965770 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 13:27:17.042949 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965773 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 13:27:17.042949 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965777 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 13:27:17.042949 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965779 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 13:27:17.042949 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965782 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 13:27:17.042949 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965785 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 13:27:17.042949 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965788 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 13:27:17.042949 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965791 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 13:27:17.042949 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965795 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 13:27:17.042949 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965798 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 13:27:17.042949 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965800 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 13:27:17.042949 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965804 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 13:27:17.042949 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965806 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 13:27:17.042949 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965809 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 13:27:17.042949 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965812 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 13:27:17.042949 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965815 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 13:27:17.042949 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965818 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 13:27:17.044685 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965821 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 13:27:17.044685 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965823 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 13:27:17.044685 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965826 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 13:27:17.044685 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965828 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 13:27:17.044685 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965831 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 13:27:17.044685 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965834 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 13:27:17.044685 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965836 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 13:27:17.044685 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965839 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 13:27:17.044685 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965842 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 13:27:17.044685 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965845 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 13:27:17.044685 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965848 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 13:27:17.044685 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965851 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 13:27:17.044685 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965856 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 13:27:17.044685 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965859 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 13:27:17.044685 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965862 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 13:27:17.044685 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965866 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 13:27:17.044685 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965870 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 13:27:17.044685 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965873 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 13:27:17.044685 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965876 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 13:27:17.044685 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965879 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 13:27:17.045571 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965881 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 13:27:17.045571 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965884 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 13:27:17.045571 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965887 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 13:27:17.045571 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965890 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 13:27:17.045571 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965893 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 13:27:17.045571 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965896 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 14 13:27:17.045571 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965899 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 13:27:17.045571 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965902 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 13:27:17.045571 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965905 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 13:27:17.045571 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965908 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 13:27:17.045571 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965911 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 13:27:17.045571 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965914 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 13:27:17.045571 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965917 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 13:27:17.045571 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965921 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 13:27:17.045571 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965923 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 13:27:17.045571 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965926 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 13:27:17.045571 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965929 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 13:27:17.045571 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965932 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 13:27:17.045571 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965935 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 13:27:17.045571 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965938 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 13:27:17.052576 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965943 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 13:27:17.052576 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965947 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 13:27:17.052576 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965950 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 13:27:17.052576 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965953 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 13:27:17.052576 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965956 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 13:27:17.052576 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965960 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 13:27:17.052576 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965963 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 13:27:17.052576 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965966 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 13:27:17.052576 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965970 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 13:27:17.052576 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965972 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 13:27:17.052576 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.965976 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 13:27:17.052576 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966784 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 13:27:17.052576 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966792 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 13:27:17.052576 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966796 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 13:27:17.052576 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966799 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 13:27:17.052576 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966801 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 13:27:17.052576 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966805 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 13:27:17.052576 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966808 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 13:27:17.053474 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966811 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 13:27:17.053474 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966814 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 13:27:17.053474 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966820 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 13:27:17.053474 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966824 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 13:27:17.053474 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966827 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 13:27:17.053474 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966830 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 13:27:17.053474 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966834 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 13:27:17.053474 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966839 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 13:27:17.053474 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966842 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 13:27:17.053474 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966845 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 13:27:17.053474 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966848 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 13:27:17.053474 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966851 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 13:27:17.053474 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966854 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 13:27:17.053474 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966857 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 13:27:17.053474 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966860 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 13:27:17.053474 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966863 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 13:27:17.053474 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966866 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 13:27:17.053474 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966869 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 13:27:17.053474 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966872 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 13:27:17.053474 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966875 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 13:27:17.056110 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966878 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 13:27:17.056110 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966881 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 13:27:17.056110 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966884 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 13:27:17.056110 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966887 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 13:27:17.056110 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966889 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 13:27:17.056110 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966892 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 13:27:17.056110 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966895 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 13:27:17.056110 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966898 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 13:27:17.056110 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966901 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 13:27:17.056110 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966904 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 13:27:17.056110 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966907 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 13:27:17.056110 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966911 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 13:27:17.056110 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966914 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 13:27:17.056110 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966917 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 13:27:17.056110 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966919 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 13:27:17.056110 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966922 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 13:27:17.056110 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966925 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 13:27:17.056110 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966928 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 13:27:17.056110 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966930 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 13:27:17.056110 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966933 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 13:27:17.056982 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966936 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 13:27:17.056982 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966939 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 13:27:17.056982 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966942 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 13:27:17.056982 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966944 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 13:27:17.056982 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966950 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 13:27:17.056982 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966953 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 13:27:17.056982 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966956 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 13:27:17.056982 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966959 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 13:27:17.056982 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966961 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 13:27:17.056982 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966966 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 13:27:17.056982 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966970 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 13:27:17.056982 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966973 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 13:27:17.056982 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966975 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 13:27:17.056982 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966978 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 13:27:17.056982 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966981 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 13:27:17.056982 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966986 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 13:27:17.056982 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966990 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 13:27:17.056982 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966993 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 13:27:17.056982 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966996 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 13:27:17.058290 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.966999 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 13:27:17.058290 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967002 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 13:27:17.058290 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967006 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 13:27:17.058290 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967009 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 13:27:17.058290 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967012 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 13:27:17.058290 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967016 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 13:27:17.058290 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967019 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 13:27:17.058290 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967022 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 13:27:17.058290 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967025 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 13:27:17.058290 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967028 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 13:27:17.058290 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967031 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 13:27:17.058290 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967035 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 13:27:17.058290 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967038 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 13:27:17.058290 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967041 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 13:27:17.058290 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967044 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 13:27:17.058290 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967047 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 13:27:17.058290 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967050 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 13:27:17.058290 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967053 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 13:27:17.058290 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967056 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 13:27:17.058290 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967058 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 13:27:17.171458 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967061 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 13:27:17.171458 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967064 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 13:27:17.171458 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967067 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 13:27:17.171458 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967070 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 13:27:17.171458 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967073 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 13:27:17.171458 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967076 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 13:27:17.171458 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967079 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 13:27:17.171458 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967081 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 13:27:17.171458 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967084 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 13:27:17.171458 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967087 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 13:27:17.171458 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967090 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 13:27:17.171458 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967092 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 13:27:17.171458 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967095 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 14 13:27:17.171458 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967560 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 13:27:17.171458 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967566 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 13:27:17.171458 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967569 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 13:27:17.171458 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967573 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 13:27:17.171458 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967576 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 13:27:17.171458 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967579 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 13:27:17.171458 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967582 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 13:27:17.172335 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967585 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 13:27:17.172335 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967588 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 13:27:17.172335 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967591 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 13:27:17.172335 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967597 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 13:27:17.172335 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967599 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 13:27:17.172335 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967603 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 13:27:17.172335 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967605 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 13:27:17.172335 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967608 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 13:27:17.172335 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967613 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 13:27:17.172335 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967617 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 13:27:17.172335 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967620 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 13:27:17.172335 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967623 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 13:27:17.172335 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967626 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 13:27:17.172335 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967629 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 13:27:17.172335 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967632 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 13:27:17.172335 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967635 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 13:27:17.172335 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967637 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 13:27:17.172335 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967640 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 13:27:17.172335 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967643 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 13:27:17.173210 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967646 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 13:27:17.173210 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967649 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 13:27:17.173210 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967652 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 13:27:17.173210 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967654 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 13:27:17.173210 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967657 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 13:27:17.173210 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967659 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 13:27:17.173210 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967662 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 13:27:17.173210 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967665 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 13:27:17.173210 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967668 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 13:27:17.173210 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967672 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 13:27:17.173210 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967674 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 13:27:17.173210 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967678 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 13:27:17.173210 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967681 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 13:27:17.173210 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967684 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 13:27:17.173210 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967686 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 13:27:17.173210 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967689 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 13:27:17.173210 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967694 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 13:27:17.173210 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967697 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 13:27:17.173210 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967699 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 13:27:17.173210 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967703 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 13:27:17.175397 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967706 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 13:27:17.175397 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967708 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 13:27:17.175397 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967711 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 13:27:17.175397 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967714 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 13:27:17.175397 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967716 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 13:27:17.175397 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967719 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 13:27:17.175397 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967722 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 13:27:17.175397 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967724 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 13:27:17.175397 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967727 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 13:27:17.175397 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967730 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 13:27:17.175397 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967732 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 13:27:17.175397 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967735 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 13:27:17.175397 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967738 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 13:27:17.175397 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967741 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 13:27:17.175397 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967743 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 13:27:17.175397 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967746 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 13:27:17.175397 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967749 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 13:27:17.175397 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967753 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 13:27:17.175397 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967756 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 13:27:17.176269 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967759 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 13:27:17.176269 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967762 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 13:27:17.176269 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967764 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 13:27:17.176269 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967767 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 13:27:17.176269 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967770 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 13:27:17.176269 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967773 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 13:27:17.176269 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967775 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 13:27:17.176269 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967778 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 13:27:17.176269 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967780 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 13:27:17.176269 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967785 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 13:27:17.176269 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967788 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 13:27:17.176269 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967790 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 13:27:17.176269 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967793 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 13:27:17.176269 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967796 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 13:27:17.176269 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967798 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 13:27:17.176269 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967801 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 13:27:17.176269 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967804 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 13:27:17.176269 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967807 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 13:27:17.176269 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967810 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 14 13:27:17.176269 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967813 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 13:27:17.177126 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967815 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 13:27:17.177126 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967818 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 13:27:17.177126 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967821 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 13:27:17.177126 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967823 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 13:27:17.177126 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967826 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 13:27:17.177126 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967829 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 13:27:17.177126 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967832 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 13:27:17.177126 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967836 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 13:27:17.177126 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967840 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 13:27:17.177126 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967843 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 13:27:17.177126 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967846 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 13:27:17.177126 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967848 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 13:27:17.177126 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967851 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 13:27:17.177126 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.967854 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 13:27:17.177126 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.967945 2583 flags.go:64] FLAG: --address="0.0.0.0" Jul 14 13:27:17.177126 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.967953 2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 14 13:27:17.177126 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.967960 2583 flags.go:64] FLAG: --anonymous-auth="true" Jul 14 13:27:17.177126 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.967966 2583 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 14 13:27:17.177126 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.967972 2583 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 14 13:27:17.177126 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.967975 2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.967980 2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.967987 2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.967991 2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.967995 2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.967999 2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968002 2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968006 2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968009 2583 flags.go:64] FLAG: --cgroup-root="" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968012 2583 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968015 2583 flags.go:64] FLAG: --client-ca-file="" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968018 2583 flags.go:64] FLAG: --cloud-provider="external" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968021 2583 flags.go:64] FLAG: --cluster-dns="[]" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968026 2583 flags.go:64] FLAG: --cluster-domain="" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968030 2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968034 2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968037 2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968041 2583 flags.go:64] FLAG: --container-log-max-files="5" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968045 2583 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968049 2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968052 2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968055 2583 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968059 2583 flags.go:64] FLAG: --contention-profiling="false" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968062 2583 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 14 13:27:17.180181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968065 2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968068 2583 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968071 2583 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968077 2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968080 2583 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968083 2583 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968086 2583 flags.go:64] FLAG: --enable-load-reader="false" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968089 2583 flags.go:64] FLAG: --enable-server="true" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968093 2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968098 2583 flags.go:64] FLAG: --event-burst="100" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968103 2583 flags.go:64] FLAG: --event-qps="50" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968106 2583 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968109 2583 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968113 2583 flags.go:64] FLAG: --eviction-hard="" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968117 2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968120 2583 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968123 2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968126 2583 flags.go:64] FLAG: --eviction-soft="" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968130 2583 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968133 2583 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968136 2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968139 2583 flags.go:64] FLAG: --experimental-mounter-path="" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968142 2583 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968145 2583 flags.go:64] FLAG: --fail-swap-on="true" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968149 2583 flags.go:64] FLAG: --feature-gates="" Jul 14 13:27:17.181016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968166 2583 flags.go:64] FLAG: --file-check-frequency="20s" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968172 2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968176 2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968179 2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968182 2583 flags.go:64] FLAG: --healthz-port="10248" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968186 2583 flags.go:64] FLAG: --help="false" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968189 2583 flags.go:64] FLAG: --hostname-override="ip-10-0-142-94.ec2.internal" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968192 2583 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968196 2583 flags.go:64] FLAG: --http-check-frequency="20s" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968199 2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968203 2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968206 2583 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968210 2583 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968213 2583 flags.go:64] FLAG: --image-service-endpoint="" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968216 2583 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968219 2583 flags.go:64] FLAG: --kube-api-burst="100" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968223 2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968228 2583 flags.go:64] FLAG: --kube-api-qps="50" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968232 2583 flags.go:64] FLAG: --kube-reserved="" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968235 2583 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968238 2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968242 2583 flags.go:64] FLAG: --kubelet-cgroups="" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968245 2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968248 2583 flags.go:64] FLAG: --lock-file="" Jul 14 13:27:17.183654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968251 2583 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968254 2583 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968257 2583 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968263 2583 flags.go:64] FLAG: --log-json-split-stream="false" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968266 2583 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968269 2583 flags.go:64] FLAG: --log-text-split-stream="false" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968272 2583 flags.go:64] FLAG: --logging-format="text" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968276 2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968279 2583 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968283 2583 flags.go:64] FLAG: --manifest-url="" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968286 2583 flags.go:64] FLAG: --manifest-url-header="" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968291 2583 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968295 2583 flags.go:64] FLAG: --max-open-files="1000000" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968300 2583 flags.go:64] FLAG: --max-pods="110" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968303 2583 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968306 2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968309 2583 flags.go:64] FLAG: --memory-manager-policy="None" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968312 2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968316 2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968319 2583 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968322 2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968331 2583 flags.go:64] FLAG: --node-status-max-images="50" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968334 2583 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968338 2583 flags.go:64] FLAG: --oom-score-adj="-999" Jul 14 13:27:17.184510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968341 2583 flags.go:64] FLAG: --pod-cidr="" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968346 2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968353 2583 flags.go:64] FLAG: --pod-manifest-path="" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968357 2583 flags.go:64] FLAG: --pod-max-pids="-1" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968360 2583 flags.go:64] FLAG: --pods-per-core="0" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968363 2583 flags.go:64] FLAG: --port="10250" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968367 2583 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968370 2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a4e1b443840117c6" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968373 2583 flags.go:64] FLAG: --qos-reserved="" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968376 2583 flags.go:64] FLAG: --read-only-port="10255" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968379 2583 flags.go:64] FLAG: --register-node="true" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968383 2583 flags.go:64] FLAG: --register-with-taints="" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968387 2583 flags.go:64] FLAG: --registry-burst="10" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968390 2583 flags.go:64] FLAG: --registry-qps="5" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968393 2583 flags.go:64] FLAG: --reserved-cpus="" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968396 2583 flags.go:64] FLAG: --reserved-memory="" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968400 2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968405 2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968408 2583 flags.go:64] FLAG: --rotate-certificates="false" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968411 2583 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968415 2583 flags.go:64] FLAG: --runonce="false" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968418 2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968421 2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968424 2583 flags.go:64] FLAG: --seccomp-default="false" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968427 2583 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 14 13:27:17.186706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968431 2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968434 2583 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968437 2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968441 2583 flags.go:64] FLAG: --storage-driver-password="root" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968443 2583 flags.go:64] FLAG: --storage-driver-secure="false" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968446 2583 flags.go:64] FLAG: --storage-driver-table="stats" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968449 2583 flags.go:64] FLAG: --storage-driver-user="root" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968453 2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968456 2583 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968461 2583 flags.go:64] FLAG: --system-cgroups="" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968464 2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968470 2583 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968473 2583 flags.go:64] FLAG: --tls-cert-file="" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968476 2583 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968481 2583 flags.go:64] FLAG: --tls-min-version="" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968484 2583 flags.go:64] FLAG: --tls-private-key-file="" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968487 2583 flags.go:64] FLAG: --topology-manager-policy="none" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968490 2583 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968493 2583 flags.go:64] FLAG: --topology-manager-scope="container" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968497 2583 flags.go:64] FLAG: --v="2" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968501 2583 flags.go:64] FLAG: --version="false" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968505 2583 flags.go:64] FLAG: --vmodule="" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968509 2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.968513 2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968644 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 13:27:17.187602 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968648 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 13:27:17.188461 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968652 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 13:27:17.188461 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968655 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 13:27:17.188461 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968657 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 13:27:17.188461 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968660 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 13:27:17.188461 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968663 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 13:27:17.188461 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968666 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 13:27:17.188461 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968669 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 13:27:17.188461 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968672 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 13:27:17.188461 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968675 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 13:27:17.188461 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968677 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 13:27:17.188461 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968680 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 13:27:17.188461 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968683 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 13:27:17.188461 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968685 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 13:27:17.188461 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968688 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 13:27:17.188461 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968691 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 13:27:17.188461 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968697 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 13:27:17.188461 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968702 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 13:27:17.188461 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968706 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 13:27:17.188461 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968709 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 13:27:17.189752 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968713 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 13:27:17.189752 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968716 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 13:27:17.189752 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968719 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 13:27:17.189752 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968722 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 13:27:17.189752 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968726 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 13:27:17.189752 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968729 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 13:27:17.189752 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968732 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 13:27:17.189752 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968735 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 13:27:17.189752 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968738 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 13:27:17.189752 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968742 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 13:27:17.189752 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968745 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 13:27:17.189752 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968749 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 13:27:17.189752 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968753 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 13:27:17.189752 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968756 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 13:27:17.189752 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968759 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 13:27:17.189752 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968763 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 13:27:17.189752 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968765 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 13:27:17.189752 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968768 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 13:27:17.189752 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968772 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 13:27:17.189752 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968775 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 13:27:17.190626 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968777 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 13:27:17.190626 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968780 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 13:27:17.190626 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968783 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 13:27:17.190626 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968786 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 13:27:17.190626 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968788 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 13:27:17.190626 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968791 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 13:27:17.190626 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968794 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 13:27:17.190626 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968797 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 13:27:17.190626 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968801 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 13:27:17.190626 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968804 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 13:27:17.190626 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968807 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 13:27:17.190626 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968811 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 13:27:17.190626 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968814 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 13:27:17.190626 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968817 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 13:27:17.190626 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968820 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 13:27:17.190626 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968823 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 13:27:17.190626 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968825 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 13:27:17.190626 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968828 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 13:27:17.190626 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968831 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 13:27:17.190626 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968834 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 13:27:17.191510 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968836 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 13:27:17.191510 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968839 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 13:27:17.191510 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968842 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 13:27:17.191510 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968846 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 13:27:17.191510 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968849 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 13:27:17.191510 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968852 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 13:27:17.191510 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968855 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 13:27:17.191510 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968858 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 13:27:17.191510 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968861 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 14 13:27:17.191510 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968864 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 13:27:17.191510 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968866 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 13:27:17.191510 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968869 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 13:27:17.191510 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968872 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 13:27:17.191510 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968875 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 13:27:17.191510 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968878 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 13:27:17.191510 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968880 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 13:27:17.191510 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968883 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 13:27:17.191510 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968886 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 13:27:17.191510 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968888 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 13:27:17.191510 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968891 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 13:27:17.192367 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968895 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 13:27:17.192367 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968898 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 13:27:17.192367 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968919 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 13:27:17.192367 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968924 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 13:27:17.192367 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968928 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 13:27:17.192367 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968931 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 13:27:17.192367 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968935 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 13:27:17.192367 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968939 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 13:27:17.192367 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968943 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 13:27:17.192367 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968947 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 13:27:17.192367 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968950 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 13:27:17.192367 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968953 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 13:27:17.192367 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968956 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 13:27:17.192367 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968960 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 13:27:17.192367 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968964 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 13:27:17.192367 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968969 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 13:27:17.192367 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968972 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 13:27:17.192367 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:16.968975 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 13:27:17.193235 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:16.969762 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 13:27:18.233121 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.233088 2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 14 13:27:18.233121 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.233117 2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 14 13:27:18.234359 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.234201 2583 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 14 13:27:18.234359 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.234213 2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 14 13:27:18.234359 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234285 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 13:27:18.234359 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234291 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 13:27:18.234359 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234295 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 13:27:18.234359 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234299 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 13:27:18.234359 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234303 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 13:27:18.234359 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234306 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 13:27:18.234359 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234309 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 13:27:18.234359 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234313 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 13:27:18.234359 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234315 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 13:27:18.234359 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234319 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 13:27:18.234359 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234321 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 13:27:18.234359 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234324 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 13:27:18.234359 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234328 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 13:27:18.234359 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234332 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 13:27:18.234359 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234334 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 13:27:18.234359 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234337 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 13:27:18.234359 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234340 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 13:27:18.234877 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234343 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 13:27:18.234877 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234346 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 13:27:18.234877 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234349 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 13:27:18.234877 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234352 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 13:27:18.234877 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234355 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 13:27:18.234877 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234358 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 13:27:18.234877 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234361 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 13:27:18.234877 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234364 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 13:27:18.234877 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234367 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 13:27:18.234877 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234370 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 13:27:18.234877 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234373 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 13:27:18.234877 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234377 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 13:27:18.234877 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234380 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 13:27:18.234877 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234383 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 13:27:18.234877 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234388 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 13:27:18.234877 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234392 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 13:27:18.234877 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234395 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 13:27:18.234877 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234398 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 13:27:18.234877 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234401 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 13:27:18.235389 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234404 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 13:27:18.235389 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234407 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 13:27:18.235389 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234410 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 13:27:18.235389 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234413 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 13:27:18.235389 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234416 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 13:27:18.235389 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234419 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 13:27:18.235389 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234422 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 13:27:18.235389 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234424 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 13:27:18.235389 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234427 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 14 13:27:18.235389 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234430 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 13:27:18.235389 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234433 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 13:27:18.235389 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234437 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 13:27:18.235389 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234440 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 13:27:18.235389 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234442 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 13:27:18.235389 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234446 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 13:27:18.235389 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234449 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 13:27:18.235389 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234452 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 13:27:18.235389 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234454 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 13:27:18.235389 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234457 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 13:27:18.235389 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234460 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 13:27:18.235905 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234463 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 13:27:18.235905 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234466 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 13:27:18.235905 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234468 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 13:27:18.235905 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234471 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 13:27:18.235905 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234474 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 13:27:18.235905 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234479 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 13:27:18.235905 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234482 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 13:27:18.235905 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234486 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 13:27:18.235905 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234491 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 13:27:18.235905 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234493 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 13:27:18.235905 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234496 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 13:27:18.235905 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234499 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 13:27:18.235905 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234502 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 13:27:18.235905 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234505 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 13:27:18.235905 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234507 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 13:27:18.235905 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234510 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 13:27:18.235905 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234513 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 13:27:18.235905 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234516 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 13:27:18.235905 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234519 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 13:27:18.236405 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234523 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 13:27:18.236405 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234525 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 13:27:18.236405 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234528 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 13:27:18.236405 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234532 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 13:27:18.236405 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234534 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 13:27:18.236405 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234537 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 13:27:18.236405 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234540 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 13:27:18.236405 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234544 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 13:27:18.236405 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234547 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 13:27:18.236405 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234550 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 13:27:18.236405 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234553 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 13:27:18.236405 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234556 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 13:27:18.236405 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234559 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 13:27:18.236405 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234562 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 13:27:18.236405 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234564 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 13:27:18.236405 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234567 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 13:27:18.236405 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234570 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 13:27:18.236405 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234573 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 13:27:18.236405 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234576 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 13:27:18.236405 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234578 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 13:27:18.236910 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234581 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 13:27:18.236910 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234584 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 13:27:18.236910 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234587 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 13:27:18.236910 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234590 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 13:27:18.236910 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.234596 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 13:27:18.236910 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234735 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 13:27:18.236910 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234742 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 13:27:18.236910 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234745 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 13:27:18.236910 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234748 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 13:27:18.236910 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234750 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 13:27:18.236910 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234753 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 13:27:18.236910 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234756 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 13:27:18.236910 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234759 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 13:27:18.236910 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234762 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 13:27:18.236910 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234765 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 13:27:18.237329 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234767 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 13:27:18.237329 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234771 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 13:27:18.237329 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234774 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 13:27:18.237329 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234777 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 13:27:18.237329 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234780 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 13:27:18.237329 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234784 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 13:27:18.237329 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234787 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 13:27:18.237329 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234790 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 13:27:18.237329 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234793 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 13:27:18.237329 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234795 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 13:27:18.237329 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234798 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 13:27:18.237329 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234801 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 13:27:18.237329 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234804 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 14 13:27:18.237329 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234807 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 13:27:18.237329 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234810 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 13:27:18.237329 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234813 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 13:27:18.237329 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234816 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 13:27:18.237329 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234818 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 13:27:18.237329 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234821 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 13:27:18.237329 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234824 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 13:27:18.237894 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234827 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 13:27:18.237894 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234829 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 13:27:18.237894 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234832 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 13:27:18.237894 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234835 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 13:27:18.237894 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234838 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 13:27:18.237894 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234840 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 13:27:18.237894 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234843 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 13:27:18.237894 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234846 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 13:27:18.237894 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234849 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 13:27:18.237894 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234852 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 13:27:18.237894 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234855 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 13:27:18.237894 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234858 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 13:27:18.237894 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234861 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 13:27:18.237894 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234863 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 13:27:18.237894 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234866 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 13:27:18.237894 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234869 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 13:27:18.237894 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234872 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 13:27:18.237894 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234875 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 13:27:18.237894 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234878 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 13:27:18.237894 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234881 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 13:27:18.238425 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234885 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 13:27:18.238425 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234888 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 13:27:18.238425 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234891 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 13:27:18.238425 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234894 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 13:27:18.238425 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234896 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 13:27:18.238425 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234899 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 13:27:18.238425 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234902 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 13:27:18.238425 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234905 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 13:27:18.238425 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234908 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 13:27:18.238425 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234910 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 13:27:18.238425 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234913 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 13:27:18.238425 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234916 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 13:27:18.238425 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234919 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 13:27:18.238425 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234923 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 13:27:18.238425 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234928 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 13:27:18.238425 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234932 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 13:27:18.238425 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234935 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 13:27:18.238425 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234938 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 13:27:18.238425 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234941 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 13:27:18.238920 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234946 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 13:27:18.238920 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234949 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 13:27:18.238920 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234952 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 13:27:18.238920 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234955 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 13:27:18.238920 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234958 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 13:27:18.238920 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234961 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 13:27:18.238920 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234964 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 13:27:18.238920 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234967 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 13:27:18.238920 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234970 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 13:27:18.238920 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234973 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 13:27:18.238920 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234977 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 13:27:18.238920 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234979 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 13:27:18.238920 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234982 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 13:27:18.238920 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234985 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 13:27:18.238920 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234988 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 13:27:18.238920 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234991 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 13:27:18.238920 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234993 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 13:27:18.238920 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.234996 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 13:27:18.238920 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.235002 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 13:27:18.238920 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.235006 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 13:27:18.239487 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.235010 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 13:27:18.239487 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.235013 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 13:27:18.239487 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.235016 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 13:27:18.239487 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.235019 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 13:27:18.239487 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.235022 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 13:27:18.239487 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.235025 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 13:27:18.239487 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.235028 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 13:27:18.239487 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.235031 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 13:27:18.239487 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.235033 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 13:27:18.239487 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:18.235036 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 13:27:18.239487 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.235041 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 13:27:18.239487 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.235203 2583 server.go:962] "Client rotation is on, will bootstrap in background" Jul 14 13:27:18.239487 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.237499 2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 14 13:27:18.239487 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.239299 2583 server.go:1019] "Starting client certificate rotation" Jul 14 13:27:18.239851 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.239415 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 13:27:18.239851 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.239472 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 13:27:18.242147 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.242132 2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 13:27:18.245388 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.245364 2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 13:27:18.254450 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.254434 2583 log.go:25] "Validated CRI v1 runtime API" Jul 14 13:27:18.258856 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.258838 2583 log.go:25] "Validated CRI v1 image API" Jul 14 13:27:18.261759 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.261736 2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 14 13:27:18.264391 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.264354 2583 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8208cfaa-47c1-4d6e-bb6d-de8be033484c:/dev/nvme0n1p4 9502c52a-41b2-49a4-8754-d15c5a03c9b1:/dev/nvme0n1p3] Jul 14 13:27:18.264470 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.264386 2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 14 13:27:18.270307 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.270184 2583 manager.go:216] Machine: {Timestamp:2026-07-14 13:27:18.268366115 +0000 UTC m=+1.795516745 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3108240 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21d7264ca2063e23582f1045ba0600 SystemUUID:ec21d726-4ca2-063e-2358-2f1045ba0600 BootID:e21a2f6d-70e1-4508-ae28-2069f1e594a9 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:34:0b:09:26:6f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:34:0b:09:26:6f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:fe:31:ec:ce:9d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 14 13:27:18.270307 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.270295 2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 14 13:27:18.270480 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.270382 2583 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 14 13:27:18.272979 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.272962 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 13:27:18.273927 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.273897 2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 14 13:27:18.274093 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.273931 2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-94.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 14 13:27:18.274735 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.274724 2583 topology_manager.go:138] "Creating topology manager with none policy" Jul 14 13:27:18.274773 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.274738 2583 container_manager_linux.go:309] "Creating device plugin manager" Jul 14 13:27:18.274773 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.274746 2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 13:27:18.276223 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.276210 2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 13:27:18.277937 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.277925 2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 14 13:27:18.281419 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.281397 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 14 13:27:18.281560 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.281548 2583 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 14 13:27:18.281620 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.281583 2583 kubelet.go:486] "Attempting to sync node with API server" Jul 14 13:27:18.281620 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.281604 2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 14 13:27:18.281700 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.281622 2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 14 13:27:18.281700 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.281633 2583 kubelet.go:398] "Adding apiserver pod source" Jul 14 13:27:18.281700 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.281644 2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 14 13:27:18.282175 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.282147 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 13:27:18.282223 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.282195 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 13:27:18.284570 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.284550 2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 14 13:27:18.286827 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.286811 2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 14 13:27:18.286905 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.286844 2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 14 13:27:18.287025 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.287015 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 14 13:27:18.287062 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.287030 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 14 13:27:18.287062 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.287037 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 14 13:27:18.287062 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.287043 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 14 13:27:18.287062 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.287052 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 14 13:27:18.287062 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.287064 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 14 13:27:18.287699 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.287689 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 14 13:27:18.287699 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.287698 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 14 13:27:18.287767 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.287706 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 14 13:27:18.287767 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.287712 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 14 13:27:18.287767 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.287730 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 14 13:27:18.287767 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.287738 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 14 13:27:18.288617 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.288606 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 14 13:27:18.288617 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.288615 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 14 13:27:18.292085 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.292069 2583 server.go:1268] "Started kubelet" Jul 14 13:27:18.292299 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.292257 2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 14 13:27:18.292369 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.292212 2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 14 13:27:18.292482 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.292456 2583 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 14 13:27:18.293016 ip-10-0-142-94 systemd[1]: Started Kubernetes Kubelet. Jul 14 13:27:18.297237 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.297190 2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 14 13:27:18.297477 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:18.297442 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-94.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 14 13:27:18.297559 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:18.297474 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 14 13:27:18.297613 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.297523 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-94.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 14 13:27:18.300394 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.300375 2583 server.go:310] "Adding debug handlers to kubelet server" Jul 14 13:27:18.300630 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:18.300602 2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 14 13:27:18.302802 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:18.301830 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-94.ec2.internal.18c22af5f708dc7f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-94.ec2.internal,UID:ip-10-0-142-94.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-94.ec2.internal,},FirstTimestamp:2026-07-14 13:27:18.292044927 +0000 UTC m=+1.819195556,LastTimestamp:2026-07-14 13:27:18.292044927 +0000 UTC m=+1.819195556,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-94.ec2.internal,}" Jul 14 13:27:18.304650 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.304632 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4pfwc" Jul 14 13:27:18.307087 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.307067 2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 14 13:27:18.307182 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.307099 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 14 13:27:18.307608 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:18.307300 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-94.ec2.internal\" not found" Jul 14 13:27:18.307786 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.307769 2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 14 13:27:18.307959 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.307902 2583 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 14 13:27:18.308046 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.308029 2583 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 14 13:27:18.308122 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.308113 2583 factory.go:55] Registering systemd factory Jul 14 13:27:18.308175 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.308129 2583 factory.go:223] Registration of the systemd container factory successfully Jul 14 13:27:18.308242 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.308229 2583 reconstruct.go:97] "Volume reconstruction finished" Jul 14 13:27:18.308337 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.308327 2583 reconciler.go:29] "Reconciler: start to sync state" Jul 14 13:27:18.308573 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.308557 2583 factory.go:153] Registering CRI-O factory Jul 14 13:27:18.308669 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.308575 2583 factory.go:223] Registration of the crio container factory successfully Jul 14 13:27:18.308669 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.308634 2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 14 13:27:18.308669 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.308659 2583 factory.go:103] Registering Raw factory Jul 14 13:27:18.308793 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.308676 2583 manager.go:1196] Started watching for new ooms in manager Jul 14 13:27:18.309473 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.309446 2583 manager.go:349] Starting recovery of all containers Jul 14 13:27:18.311590 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:18.311560 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 14 13:27:18.311942 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:18.311915 2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-94.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 14 13:27:18.313635 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.313609 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4pfwc" Jul 14 13:27:18.321560 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.321544 2583 manager.go:354] Recovery completed Jul 14 13:27:18.325978 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.325962 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:27:18.330342 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.330269 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:27:18.330342 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.330309 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:27:18.330342 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.330329 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:27:18.331119 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.330969 2583 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 14 13:27:18.331236 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.331136 2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 14 13:27:18.331236 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.331153 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 14 13:27:18.331236 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.330974 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:27:18.332498 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.332482 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:27:18.332568 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.332511 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:27:18.332568 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.332522 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:27:18.332680 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.332595 2583 policy_none.go:49] "None policy: Start" Jul 14 13:27:18.332680 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.332609 2583 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 14 13:27:18.332680 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.332630 2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 14 13:27:18.333901 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.333881 2583 policy_none.go:47] "Start" Jul 14 13:27:18.334052 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:18.333875 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-94.ec2.internal.18c22af5f9506d6b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-94.ec2.internal,UID:ip-10-0-142-94.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-94.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-94.ec2.internal,},FirstTimestamp:2026-07-14 13:27:18.330289515 +0000 UTC m=+1.857440146,LastTimestamp:2026-07-14 13:27:18.330289515 +0000 UTC m=+1.857440146,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-94.ec2.internal,}" Jul 14 13:27:18.368763 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.368745 2583 manager.go:337] "Starting Device Plugin manager" Jul 14 13:27:18.368884 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:18.368782 2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 14 13:27:18.368884 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.368793 2583 server.go:87] "Starting device plugin registration server" Jul 14 13:27:18.369120 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.369094 2583 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 14 13:27:18.369174 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.369139 2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 14 13:27:18.369290 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.369276 2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 14 13:27:18.369368 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.369354 2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 14 13:27:18.369410 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.369369 2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 14 13:27:18.369910 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:18.369889 2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 14 13:27:18.369996 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:18.369940 2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-94.ec2.internal\" not found" Jul 14 13:27:18.469353 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.469314 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:27:18.471164 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.471140 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:27:18.471259 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.471175 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:27:18.471259 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.471189 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:27:18.471259 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.471224 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-94.ec2.internal" Jul 14 13:27:18.484898 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.484875 2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-94.ec2.internal" Jul 14 13:27:18.484999 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:18.484901 2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-94.ec2.internal\": node \"ip-10-0-142-94.ec2.internal\" not found" Jul 14 13:27:18.485076 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.485058 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 14 13:27:18.486507 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.486488 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 14 13:27:18.486605 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.486528 2583 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 14 13:27:18.486605 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.486552 2583 kubelet.go:2448] "Starting kubelet main sync loop" Jul 14 13:27:18.486605 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:18.486585 2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 14 13:27:18.488954 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.488932 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 13:27:18.543895 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:18.543861 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-94.ec2.internal\" not found" Jul 14 13:27:18.589049 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.588955 2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-94.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-94.ec2.internal"] Jul 14 13:27:18.589230 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.589059 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:27:18.590795 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.590777 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:27:18.590920 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.590809 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:27:18.590920 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.590820 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:27:18.592489 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.592474 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:27:18.592619 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.592603 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-94.ec2.internal" Jul 14 13:27:18.592682 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.592646 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:27:18.593898 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.593881 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:27:18.593968 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.593914 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:27:18.593968 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.593926 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:27:18.593968 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.593881 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:27:18.594062 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.594010 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:27:18.594062 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.594026 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:27:18.595655 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.595634 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-94.ec2.internal" Jul 14 13:27:18.595746 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.595668 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:27:18.599087 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.599070 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:27:18.599186 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.599097 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:27:18.599186 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.599109 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:27:18.611351 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.611331 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e88c1491a47c596d63c06a067a091e22-config\") pod \"kube-apiserver-proxy-ip-10-0-142-94.ec2.internal\" (UID: \"e88c1491a47c596d63c06a067a091e22\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-94.ec2.internal" Jul 14 13:27:18.611418 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.611359 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2adea4ae4de3c3672673fba8673389d5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-94.ec2.internal\" (UID: \"2adea4ae4de3c3672673fba8673389d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-94.ec2.internal" Jul 14 13:27:18.611418 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.611379 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2adea4ae4de3c3672673fba8673389d5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-94.ec2.internal\" (UID: \"2adea4ae4de3c3672673fba8673389d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-94.ec2.internal" Jul 14 13:27:18.618851 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:18.618830 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-94.ec2.internal\" not found" node="ip-10-0-142-94.ec2.internal" Jul 14 13:27:18.624032 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:18.624012 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-94.ec2.internal\" not found" node="ip-10-0-142-94.ec2.internal" Jul 14 13:27:18.644086 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:18.644059 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-94.ec2.internal\" not found" Jul 14 13:27:18.711638 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.711603 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2adea4ae4de3c3672673fba8673389d5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-94.ec2.internal\" (UID: \"2adea4ae4de3c3672673fba8673389d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-94.ec2.internal" Jul 14 13:27:18.711638 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.711643 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2adea4ae4de3c3672673fba8673389d5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-94.ec2.internal\" (UID: \"2adea4ae4de3c3672673fba8673389d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-94.ec2.internal" Jul 14 13:27:18.711790 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.711662 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e88c1491a47c596d63c06a067a091e22-config\") pod \"kube-apiserver-proxy-ip-10-0-142-94.ec2.internal\" (UID: \"e88c1491a47c596d63c06a067a091e22\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-94.ec2.internal" Jul 14 13:27:18.711790 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.711723 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2adea4ae4de3c3672673fba8673389d5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-94.ec2.internal\" (UID: \"2adea4ae4de3c3672673fba8673389d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-94.ec2.internal" Jul 14 13:27:18.711922 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.711789 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e88c1491a47c596d63c06a067a091e22-config\") pod \"kube-apiserver-proxy-ip-10-0-142-94.ec2.internal\" (UID: \"e88c1491a47c596d63c06a067a091e22\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-94.ec2.internal" Jul 14 13:27:18.711922 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.711823 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2adea4ae4de3c3672673fba8673389d5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-94.ec2.internal\" (UID: \"2adea4ae4de3c3672673fba8673389d5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-94.ec2.internal" Jul 14 13:27:18.744512 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:18.744480 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-94.ec2.internal\" not found" Jul 14 13:27:18.846807 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:18.846700 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-94.ec2.internal\" not found" Jul 14 13:27:18.921871 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.921833 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-94.ec2.internal" Jul 14 13:27:18.927590 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:18.927574 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-94.ec2.internal" Jul 14 13:27:18.947792 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:18.947764 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-94.ec2.internal\" not found" Jul 14 13:27:19.049717 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:19.049668 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-94.ec2.internal\" not found" Jul 14 13:27:19.150510 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:19.150409 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-94.ec2.internal\" not found" Jul 14 13:27:19.238262 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:19.238214 2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 14 13:27:19.238661 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:19.238456 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 13:27:19.251322 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:19.251296 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-94.ec2.internal\" not found" Jul 14 13:27:19.317958 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:19.317915 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 13:22:18 +0000 UTC" deadline="2027-12-27 06:22:46.817365971 +0000 UTC" Jul 14 13:27:19.317958 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:19.317949 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12736h55m27.499420438s" Jul 14 13:27:19.351455 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:19.351416 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-94.ec2.internal\" not found" Jul 14 13:27:19.454552 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:19.454453 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-94.ec2.internal\" not found" Jul 14 13:27:19.555854 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:19.555801 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-94.ec2.internal\" not found" Jul 14 13:27:19.659201 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:19.659143 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-94.ec2.internal\" not found" Jul 14 13:27:19.705513 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:19.705411 2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 13:27:19.758626 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:19.758600 2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 13:27:19.759609 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:19.759590 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-94.ec2.internal\" not found" Jul 14 13:27:19.798850 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:19.798825 2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 13:27:19.807816 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:19.807787 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-94.ec2.internal" Jul 14 13:27:19.828215 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:19.828188 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 13:27:19.829613 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:19.829597 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-94.ec2.internal" Jul 14 13:27:19.846748 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:19.846717 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 13:27:20.015377 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:20.015336 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode88c1491a47c596d63c06a067a091e22.slice/crio-dcc4860cb65c5b2d7bdf89599917ec3cd8842dd5eaa25686f4ef90d95b107fc7 WatchSource:0}: Error finding container dcc4860cb65c5b2d7bdf89599917ec3cd8842dd5eaa25686f4ef90d95b107fc7: Status 404 returned error can't find the container with id dcc4860cb65c5b2d7bdf89599917ec3cd8842dd5eaa25686f4ef90d95b107fc7 Jul 14 13:27:20.015901 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:20.015876 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2adea4ae4de3c3672673fba8673389d5.slice/crio-d004d0a5edc0d98b83f2504b0bba23fc8d297e23cf3dad24f79c439fff7e0a8d WatchSource:0}: Error finding container d004d0a5edc0d98b83f2504b0bba23fc8d297e23cf3dad24f79c439fff7e0a8d: Status 404 returned error can't find the container with id d004d0a5edc0d98b83f2504b0bba23fc8d297e23cf3dad24f79c439fff7e0a8d Jul 14 13:27:20.020870 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.020842 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 13:27:20.046348 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.046308 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 13:27:20.283288 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.283191 2583 apiserver.go:52] "Watching apiserver" Jul 14 13:27:20.296097 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.296067 2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 14 13:27:20.298331 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.298305 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-dqvh8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-94.ec2.internal","openshift-network-diagnostics/network-check-target-wnnf2","openshift-network-operator/iptables-alerter-vjxkn","openshift-ovn-kubernetes/ovnkube-node-lpnff","kube-system/global-pull-secret-syncer-z9kpt","kube-system/kube-apiserver-proxy-ip-10-0-142-94.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm","openshift-cluster-node-tuning-operator/tuned-8hmpf","openshift-dns/node-resolver-plk87","openshift-multus/multus-7fmx5","openshift-multus/multus-additional-cni-plugins-xzjv6","openshift-multus/network-metrics-daemon-xrgrj","kube-system/konnectivity-agent-k24pk"] Jul 14 13:27:20.300804 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.300784 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-plk87" Jul 14 13:27:20.302194 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.302166 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:20.302299 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:20.302241 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrgrj" podUID="50a391d6-3c2b-478d-b005-4bbd65bdec74" Jul 14 13:27:20.303437 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.303416 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:20.303522 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:20.303492 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnnf2" podUID="1bf7c74f-a569-44e9-98cb-ee2255c6067e" Jul 14 13:27:20.304560 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.304543 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kj27l\"" Jul 14 13:27:20.304633 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.304581 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vjxkn" Jul 14 13:27:20.304799 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.304700 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 14 13:27:20.305666 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.305650 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 14 13:27:20.306483 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.306466 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.307626 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.307451 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 14 13:27:20.308535 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.308513 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:20.308638 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:20.308586 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9kpt" podUID="92645e2c-0c99-4eca-930a-686d53f52229" Jul 14 13:27:20.309994 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.309973 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.311167 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.310975 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 14 13:27:20.311167 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.311109 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 14 13:27:20.311332 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.311277 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 14 13:27:20.311332 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.311284 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 14 13:27:20.311332 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.311305 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 14 13:27:20.311495 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.311355 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 14 13:27:20.311495 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.311434 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 14 13:27:20.312335 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.312317 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 14 13:27:20.312628 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.312613 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k24pk" Jul 14 13:27:20.312890 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.312867 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cstbr\"" Jul 14 13:27:20.312890 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.312881 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jz5kp\"" Jul 14 13:27:20.314488 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.313963 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 14 13:27:20.315724 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.314635 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xh88p\"" Jul 14 13:27:20.315724 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.314763 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.316752 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.316699 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 14 13:27:20.316861 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.316814 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 14 13:27:20.317944 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.317924 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dqvh8" Jul 14 13:27:20.319828 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.319811 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.320502 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320466 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmfpx\" (UniqueName: \"kubernetes.io/projected/02bc37b4-5dba-4801-92af-07beda478f46-kube-api-access-gmfpx\") pod \"node-resolver-plk87\" (UID: \"02bc37b4-5dba-4801-92af-07beda478f46\") " pod="openshift-dns/node-resolver-plk87" Jul 14 13:27:20.320585 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320513 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cs9c\" (UniqueName: \"kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c\") pod \"network-check-target-wnnf2\" (UID: \"1bf7c74f-a569-44e9-98cb-ee2255c6067e\") " pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:20.320585 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320540 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-etc-openvswitch\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.320585 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320568 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-run-openvswitch\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.320698 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320590 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/02bc37b4-5dba-4801-92af-07beda478f46-hosts-file\") pod \"node-resolver-plk87\" (UID: \"02bc37b4-5dba-4801-92af-07beda478f46\") " pod="openshift-dns/node-resolver-plk87" Jul 14 13:27:20.320698 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320606 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-host-cni-bin\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.320698 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320631 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.320698 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320650 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret\") pod \"global-pull-secret-syncer-z9kpt\" (UID: \"92645e2c-0c99-4eca-930a-686d53f52229\") " pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:20.320698 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320683 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2f52c389-0177-4d3b-93f4-977293414fc1-iptables-alerter-script\") pod \"iptables-alerter-vjxkn\" (UID: \"2f52c389-0177-4d3b-93f4-977293414fc1\") " pod="openshift-network-operator/iptables-alerter-vjxkn" Jul 14 13:27:20.320888 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320710 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-host-kubelet\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.320888 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320729 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-run-ovn\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.320888 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320747 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.320888 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320773 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/694177f0-32e8-4724-b626-ca46d6c476b5-ovn-node-metrics-cert\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.320888 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320804 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/92645e2c-0c99-4eca-930a-686d53f52229-dbus\") pod \"global-pull-secret-syncer-z9kpt\" (UID: \"92645e2c-0c99-4eca-930a-686d53f52229\") " pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:20.320888 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320823 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs\") pod \"network-metrics-daemon-xrgrj\" (UID: \"50a391d6-3c2b-478d-b005-4bbd65bdec74\") " pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:20.320888 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320846 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/92645e2c-0c99-4eca-930a-686d53f52229-kubelet-config\") pod \"global-pull-secret-syncer-z9kpt\" (UID: \"92645e2c-0c99-4eca-930a-686d53f52229\") " pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:20.320888 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320864 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx2vd\" (UniqueName: \"kubernetes.io/projected/2f52c389-0177-4d3b-93f4-977293414fc1-kube-api-access-kx2vd\") pod \"iptables-alerter-vjxkn\" (UID: \"2f52c389-0177-4d3b-93f4-977293414fc1\") " pod="openshift-network-operator/iptables-alerter-vjxkn" Jul 14 13:27:20.320888 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320880 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-host-slash\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.321231 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320899 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/694177f0-32e8-4724-b626-ca46d6c476b5-env-overrides\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.321231 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320916 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rg5m\" (UniqueName: \"kubernetes.io/projected/50a391d6-3c2b-478d-b005-4bbd65bdec74-kube-api-access-9rg5m\") pod \"network-metrics-daemon-xrgrj\" (UID: \"50a391d6-3c2b-478d-b005-4bbd65bdec74\") " pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:20.321231 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320962 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-systemd-units\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.321231 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.320984 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-run-systemd\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.321231 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.321007 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-host-cni-netd\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.321231 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.321022 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/694177f0-32e8-4724-b626-ca46d6c476b5-ovnkube-config\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.321231 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.321043 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-node-log\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.321231 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.321059 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2f52c389-0177-4d3b-93f4-977293414fc1-host-slash\") pod \"iptables-alerter-vjxkn\" (UID: \"2f52c389-0177-4d3b-93f4-977293414fc1\") " pod="openshift-network-operator/iptables-alerter-vjxkn" Jul 14 13:27:20.321231 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.321075 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj6nm\" (UniqueName: \"kubernetes.io/projected/694177f0-32e8-4724-b626-ca46d6c476b5-kube-api-access-xj6nm\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.321231 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.321118 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-host-run-netns\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.321231 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.321133 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-var-lib-openvswitch\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.321231 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.321149 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-log-socket\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.321231 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.321163 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/694177f0-32e8-4724-b626-ca46d6c476b5-ovnkube-script-lib\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.321231 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.321178 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02bc37b4-5dba-4801-92af-07beda478f46-tmp-dir\") pod \"node-resolver-plk87\" (UID: \"02bc37b4-5dba-4801-92af-07beda478f46\") " pod="openshift-dns/node-resolver-plk87" Jul 14 13:27:20.321855 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.321350 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 14 13:27:20.321855 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.321358 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 14 13:27:20.321855 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.321641 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.326232 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.326211 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 14 13:27:20.328087 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.328064 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 14 13:27:20.328202 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.328166 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bksxw\"" Jul 14 13:27:20.328202 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.328178 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 14 13:27:20.328758 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.328738 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 14 13:27:20.328825 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.328766 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ts6z6\"" Jul 14 13:27:20.328881 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.328768 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 14 13:27:20.329006 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.328982 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 14 13:27:20.329149 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.329123 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 14 13:27:20.329149 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.329133 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 14 13:27:20.329503 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.329480 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 14 13:27:20.329503 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.329502 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 14 13:27:20.329664 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.329511 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r5cv6\"" Jul 14 13:27:20.329664 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.329528 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-25p99\"" Jul 14 13:27:20.329745 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.329677 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 14 13:27:20.329745 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.329687 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 14 13:27:20.329745 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.329711 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fwrqk\"" Jul 14 13:27:20.348173 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.348140 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 13:27:20.368094 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.368062 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cms2k" Jul 14 13:27:20.384241 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.384212 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cms2k" Jul 14 13:27:20.409438 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.409407 2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 14 13:27:20.422393 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.422369 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-var-lib-kubelet\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.422482 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.422400 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvrjz\" (UniqueName: \"kubernetes.io/projected/6aec7895-aa6e-40c8-95ec-2b61bf996da0-kube-api-access-pvrjz\") pod \"node-ca-dqvh8\" (UID: \"6aec7895-aa6e-40c8-95ec-2b61bf996da0\") " pod="openshift-image-registry/node-ca-dqvh8" Jul 14 13:27:20.422482 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.422425 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/56dc0a37-3391-46e8-b21d-71b167fd843f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.422482 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.422461 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/86df2bc2-9305-4329-8bc4-53abd12c87ff-cni-binary-copy\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.422583 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.422506 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx2vd\" (UniqueName: \"kubernetes.io/projected/2f52c389-0177-4d3b-93f4-977293414fc1-kube-api-access-kx2vd\") pod \"iptables-alerter-vjxkn\" (UID: \"2f52c389-0177-4d3b-93f4-977293414fc1\") " pod="openshift-network-operator/iptables-alerter-vjxkn" Jul 14 13:27:20.422583 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.422535 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/694177f0-32e8-4724-b626-ca46d6c476b5-env-overrides\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.422692 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.422668 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-system-cni-dir\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.422817 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.422798 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-etc-tuned\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.422855 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.422830 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4bc5e553-528f-46c0-8724-20c8c6d8024c-konnectivity-ca\") pod \"konnectivity-agent-k24pk\" (UID: \"4bc5e553-528f-46c0-8724-20c8c6d8024c\") " pod="kube-system/konnectivity-agent-k24pk" Jul 14 13:27:20.422889 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.422881 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-systemd-units\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.422961 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.422945 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-host-var-lib-cni-bin\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.423010 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.422965 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-systemd-units\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.423073 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423044 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-etc-sysctl-conf\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.423153 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423120 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56dc0a37-3391-46e8-b21d-71b167fd843f-cni-binary-copy\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.423215 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423163 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/602e589c-c901-49a2-b9b8-390571f5e5c4-sys-fs\") pod \"aws-ebs-csi-driver-node-pjznm\" (UID: \"602e589c-c901-49a2-b9b8-390571f5e5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.423215 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423191 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-cnibin\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.423324 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423219 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-host-run-k8s-cni-cncf-io\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.423324 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423250 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/602e589c-c901-49a2-b9b8-390571f5e5c4-device-dir\") pod \"aws-ebs-csi-driver-node-pjznm\" (UID: \"602e589c-c901-49a2-b9b8-390571f5e5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.423324 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423277 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-multus-socket-dir-parent\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.423324 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423309 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj6nm\" (UniqueName: \"kubernetes.io/projected/694177f0-32e8-4724-b626-ca46d6c476b5-kube-api-access-xj6nm\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.423524 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423336 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-host-run-netns\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.423524 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423363 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-var-lib-openvswitch\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.423524 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423391 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-log-socket\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.423524 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423413 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-host-run-netns\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.423524 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423418 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02bc37b4-5dba-4801-92af-07beda478f46-tmp-dir\") pod \"node-resolver-plk87\" (UID: \"02bc37b4-5dba-4801-92af-07beda478f46\") " pod="openshift-dns/node-resolver-plk87" Jul 14 13:27:20.423524 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423471 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/602e589c-c901-49a2-b9b8-390571f5e5c4-socket-dir\") pod \"aws-ebs-csi-driver-node-pjznm\" (UID: \"602e589c-c901-49a2-b9b8-390571f5e5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.423524 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423489 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-var-lib-openvswitch\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.423524 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423497 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-host-run-netns\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.423948 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423526 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/56dc0a37-3391-46e8-b21d-71b167fd843f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.423948 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423550 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-log-socket\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.423948 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423575 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-etc-openvswitch\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.423948 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423627 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqbtl\" (UniqueName: \"kubernetes.io/projected/602e589c-c901-49a2-b9b8-390571f5e5c4-kube-api-access-vqbtl\") pod \"aws-ebs-csi-driver-node-pjznm\" (UID: \"602e589c-c901-49a2-b9b8-390571f5e5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.423948 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423657 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56dc0a37-3391-46e8-b21d-71b167fd843f-cnibin\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.423948 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423710 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbr98\" (UniqueName: \"kubernetes.io/projected/56dc0a37-3391-46e8-b21d-71b167fd843f-kube-api-access-jbr98\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.423948 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423739 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-etc-openvswitch\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.423948 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423753 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02bc37b4-5dba-4801-92af-07beda478f46-tmp-dir\") pod \"node-resolver-plk87\" (UID: \"02bc37b4-5dba-4801-92af-07beda478f46\") " pod="openshift-dns/node-resolver-plk87" Jul 14 13:27:20.423948 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423768 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/694177f0-32e8-4724-b626-ca46d6c476b5-env-overrides\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.423948 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423813 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4bc5e553-528f-46c0-8724-20c8c6d8024c-agent-certs\") pod \"konnectivity-agent-k24pk\" (UID: \"4bc5e553-528f-46c0-8724-20c8c6d8024c\") " pod="kube-system/konnectivity-agent-k24pk" Jul 14 13:27:20.423948 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423848 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-host-cni-bin\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.423948 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423873 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/602e589c-c901-49a2-b9b8-390571f5e5c4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pjznm\" (UID: \"602e589c-c901-49a2-b9b8-390571f5e5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.423948 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423898 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/602e589c-c901-49a2-b9b8-390571f5e5c4-registration-dir\") pod \"aws-ebs-csi-driver-node-pjznm\" (UID: \"602e589c-c901-49a2-b9b8-390571f5e5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.423948 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423914 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-host-cni-bin\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.423948 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423923 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-os-release\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.423948 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423947 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-etc-systemd\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.424721 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.423969 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6aec7895-aa6e-40c8-95ec-2b61bf996da0-serviceca\") pod \"node-ca-dqvh8\" (UID: \"6aec7895-aa6e-40c8-95ec-2b61bf996da0\") " pod="openshift-image-registry/node-ca-dqvh8" Jul 14 13:27:20.424721 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424007 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2f52c389-0177-4d3b-93f4-977293414fc1-iptables-alerter-script\") pod \"iptables-alerter-vjxkn\" (UID: \"2f52c389-0177-4d3b-93f4-977293414fc1\") " pod="openshift-network-operator/iptables-alerter-vjxkn" Jul 14 13:27:20.424721 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424034 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-host-kubelet\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.424721 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424060 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-run-ovn\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.424721 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424084 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/694177f0-32e8-4724-b626-ca46d6c476b5-ovn-node-metrics-cert\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.424721 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424093 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-host-kubelet\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.424721 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424109 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/92645e2c-0c99-4eca-930a-686d53f52229-dbus\") pod \"global-pull-secret-syncer-z9kpt\" (UID: \"92645e2c-0c99-4eca-930a-686d53f52229\") " pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:20.424721 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424135 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-etc-kubernetes\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.424721 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424137 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-run-ovn\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.424721 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424186 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/92645e2c-0c99-4eca-930a-686d53f52229-kubelet-config\") pod \"global-pull-secret-syncer-z9kpt\" (UID: \"92645e2c-0c99-4eca-930a-686d53f52229\") " pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:20.424721 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424233 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-host-slash\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.424721 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424267 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rg5m\" (UniqueName: \"kubernetes.io/projected/50a391d6-3c2b-478d-b005-4bbd65bdec74-kube-api-access-9rg5m\") pod \"network-metrics-daemon-xrgrj\" (UID: \"50a391d6-3c2b-478d-b005-4bbd65bdec74\") " pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:20.424721 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424283 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/92645e2c-0c99-4eca-930a-686d53f52229-dbus\") pod \"global-pull-secret-syncer-z9kpt\" (UID: \"92645e2c-0c99-4eca-930a-686d53f52229\") " pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:20.424721 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424295 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-multus-cni-dir\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.424721 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424305 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/92645e2c-0c99-4eca-930a-686d53f52229-kubelet-config\") pod \"global-pull-secret-syncer-z9kpt\" (UID: \"92645e2c-0c99-4eca-930a-686d53f52229\") " pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:20.424721 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424334 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r52v\" (UniqueName: \"kubernetes.io/projected/86df2bc2-9305-4329-8bc4-53abd12c87ff-kube-api-access-9r52v\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.424721 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424344 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-host-slash\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.425417 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424367 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-etc-modprobe-d\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.425417 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424394 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-sys\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.425417 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424421 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-run-systemd\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.425417 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424448 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-host-cni-netd\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.425417 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424475 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-run-systemd\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.425417 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424473 2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 14 13:27:20.425417 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424476 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/694177f0-32e8-4724-b626-ca46d6c476b5-ovnkube-config\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.425417 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424506 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-host-cni-netd\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.425417 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424518 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-etc-sysconfig\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.425417 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424566 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-node-log\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.425417 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424592 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-host-var-lib-kubelet\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.425417 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424614 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/56dc0a37-3391-46e8-b21d-71b167fd843f-os-release\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.425417 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424643 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2f52c389-0177-4d3b-93f4-977293414fc1-iptables-alerter-script\") pod \"iptables-alerter-vjxkn\" (UID: \"2f52c389-0177-4d3b-93f4-977293414fc1\") " pod="openshift-network-operator/iptables-alerter-vjxkn" Jul 14 13:27:20.425417 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424654 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2f52c389-0177-4d3b-93f4-977293414fc1-host-slash\") pod \"iptables-alerter-vjxkn\" (UID: \"2f52c389-0177-4d3b-93f4-977293414fc1\") " pod="openshift-network-operator/iptables-alerter-vjxkn" Jul 14 13:27:20.425417 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424646 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-node-log\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.425417 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424677 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-hostroot\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.425417 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424702 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-host-run-multus-certs\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.425417 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424702 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2f52c389-0177-4d3b-93f4-977293414fc1-host-slash\") pod \"iptables-alerter-vjxkn\" (UID: \"2f52c389-0177-4d3b-93f4-977293414fc1\") " pod="openshift-network-operator/iptables-alerter-vjxkn" Jul 14 13:27:20.426261 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424751 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-lib-modules\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.426261 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424775 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h696j\" (UniqueName: \"kubernetes.io/projected/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-kube-api-access-h696j\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.426261 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424796 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6aec7895-aa6e-40c8-95ec-2b61bf996da0-host\") pod \"node-ca-dqvh8\" (UID: \"6aec7895-aa6e-40c8-95ec-2b61bf996da0\") " pod="openshift-image-registry/node-ca-dqvh8" Jul 14 13:27:20.426261 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424822 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/694177f0-32e8-4724-b626-ca46d6c476b5-ovnkube-script-lib\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.426261 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424869 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmfpx\" (UniqueName: \"kubernetes.io/projected/02bc37b4-5dba-4801-92af-07beda478f46-kube-api-access-gmfpx\") pod \"node-resolver-plk87\" (UID: \"02bc37b4-5dba-4801-92af-07beda478f46\") " pod="openshift-dns/node-resolver-plk87" Jul 14 13:27:20.426261 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424908 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cs9c\" (UniqueName: \"kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c\") pod \"network-check-target-wnnf2\" (UID: \"1bf7c74f-a569-44e9-98cb-ee2255c6067e\") " pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:20.426261 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424940 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-run-openvswitch\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.426261 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424959 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/02bc37b4-5dba-4801-92af-07beda478f46-hosts-file\") pod \"node-resolver-plk87\" (UID: \"02bc37b4-5dba-4801-92af-07beda478f46\") " pod="openshift-dns/node-resolver-plk87" Jul 14 13:27:20.426261 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.424986 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/602e589c-c901-49a2-b9b8-390571f5e5c4-etc-selinux\") pod \"aws-ebs-csi-driver-node-pjznm\" (UID: \"602e589c-c901-49a2-b9b8-390571f5e5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.426261 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.425011 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-tmp\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.426261 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.425028 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/694177f0-32e8-4724-b626-ca46d6c476b5-ovnkube-config\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.426261 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.425037 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56dc0a37-3391-46e8-b21d-71b167fd843f-system-cni-dir\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.426261 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.425046 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-run-openvswitch\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.426261 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.425067 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/02bc37b4-5dba-4801-92af-07beda478f46-hosts-file\") pod \"node-resolver-plk87\" (UID: \"02bc37b4-5dba-4801-92af-07beda478f46\") " pod="openshift-dns/node-resolver-plk87" Jul 14 13:27:20.426261 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.425101 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.426261 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.425131 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret\") pod \"global-pull-secret-syncer-z9kpt\" (UID: \"92645e2c-0c99-4eca-930a-686d53f52229\") " pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:20.426261 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.425160 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-multus-conf-dir\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.427074 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.425173 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.427074 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.425184 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/86df2bc2-9305-4329-8bc4-53abd12c87ff-multus-daemon-config\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.427074 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.425210 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-etc-kubernetes\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.427074 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.425234 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-etc-sysctl-d\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.427074 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.425255 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-host\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.427074 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.425271 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56dc0a37-3391-46e8-b21d-71b167fd843f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.427074 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:20.425282 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:20.427074 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.425295 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.427074 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.425322 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs\") pod \"network-metrics-daemon-xrgrj\" (UID: \"50a391d6-3c2b-478d-b005-4bbd65bdec74\") " pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:20.427074 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.425338 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-host-var-lib-cni-multus\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.427074 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.425354 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/694177f0-32e8-4724-b626-ca46d6c476b5-ovnkube-script-lib\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.427074 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:20.425387 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret podName:92645e2c-0c99-4eca-930a-686d53f52229 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:20.925350009 +0000 UTC m=+4.452500643 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret") pod "global-pull-secret-syncer-z9kpt" (UID: "92645e2c-0c99-4eca-930a-686d53f52229") : object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:20.427074 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.425398 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/694177f0-32e8-4724-b626-ca46d6c476b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.427074 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.425409 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-run\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.427074 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:20.425429 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:20.427074 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:20.425498 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs podName:50a391d6-3c2b-478d-b005-4bbd65bdec74 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:20.925484521 +0000 UTC m=+4.452635137 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs") pod "network-metrics-daemon-xrgrj" (UID: "50a391d6-3c2b-478d-b005-4bbd65bdec74") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:20.428924 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.428880 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/694177f0-32e8-4724-b626-ca46d6c476b5-ovn-node-metrics-cert\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.434279 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.434235 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj6nm\" (UniqueName: \"kubernetes.io/projected/694177f0-32e8-4724-b626-ca46d6c476b5-kube-api-access-xj6nm\") pod \"ovnkube-node-lpnff\" (UID: \"694177f0-32e8-4724-b626-ca46d6c476b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.439648 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.439611 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx2vd\" (UniqueName: \"kubernetes.io/projected/2f52c389-0177-4d3b-93f4-977293414fc1-kube-api-access-kx2vd\") pod \"iptables-alerter-vjxkn\" (UID: \"2f52c389-0177-4d3b-93f4-977293414fc1\") " pod="openshift-network-operator/iptables-alerter-vjxkn" Jul 14 13:27:20.444255 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:20.444234 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 13:27:20.444371 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:20.444261 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 13:27:20.444371 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:20.444291 2583 projected.go:196] Error preparing data for projected volume kube-api-access-2cs9c for pod openshift-network-diagnostics/network-check-target-wnnf2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:20.444476 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:20.444371 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c podName:1bf7c74f-a569-44e9-98cb-ee2255c6067e nodeName:}" failed. No retries permitted until 2026-07-14 13:27:20.944352126 +0000 UTC m=+4.471502758 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2cs9c" (UniqueName: "kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c") pod "network-check-target-wnnf2" (UID: "1bf7c74f-a569-44e9-98cb-ee2255c6067e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:20.447526 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.447505 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmfpx\" (UniqueName: \"kubernetes.io/projected/02bc37b4-5dba-4801-92af-07beda478f46-kube-api-access-gmfpx\") pod \"node-resolver-plk87\" (UID: \"02bc37b4-5dba-4801-92af-07beda478f46\") " pod="openshift-dns/node-resolver-plk87" Jul 14 13:27:20.449650 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.449618 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rg5m\" (UniqueName: \"kubernetes.io/projected/50a391d6-3c2b-478d-b005-4bbd65bdec74-kube-api-access-9rg5m\") pod \"network-metrics-daemon-xrgrj\" (UID: \"50a391d6-3c2b-478d-b005-4bbd65bdec74\") " pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:20.491589 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.491525 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-94.ec2.internal" event={"ID":"2adea4ae4de3c3672673fba8673389d5","Type":"ContainerStarted","Data":"d004d0a5edc0d98b83f2504b0bba23fc8d297e23cf3dad24f79c439fff7e0a8d"} Jul 14 13:27:20.492861 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.492833 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-94.ec2.internal" event={"ID":"e88c1491a47c596d63c06a067a091e22","Type":"ContainerStarted","Data":"dcc4860cb65c5b2d7bdf89599917ec3cd8842dd5eaa25686f4ef90d95b107fc7"} Jul 14 13:27:20.526436 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526406 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-host-var-lib-cni-bin\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.526624 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526443 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-etc-sysctl-conf\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.526624 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526470 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56dc0a37-3391-46e8-b21d-71b167fd843f-cni-binary-copy\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.526624 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526493 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/602e589c-c901-49a2-b9b8-390571f5e5c4-sys-fs\") pod \"aws-ebs-csi-driver-node-pjznm\" (UID: \"602e589c-c901-49a2-b9b8-390571f5e5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.526624 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526535 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-host-var-lib-cni-bin\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.526624 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526517 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-cnibin\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.526624 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526605 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-cnibin\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.526624 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526606 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-host-run-k8s-cni-cncf-io\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.526989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526637 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-host-run-k8s-cni-cncf-io\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.526989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526651 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/602e589c-c901-49a2-b9b8-390571f5e5c4-device-dir\") pod \"aws-ebs-csi-driver-node-pjznm\" (UID: \"602e589c-c901-49a2-b9b8-390571f5e5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.526989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526654 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-etc-sysctl-conf\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.526989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526675 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-multus-socket-dir-parent\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.526989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526693 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/602e589c-c901-49a2-b9b8-390571f5e5c4-sys-fs\") pod \"aws-ebs-csi-driver-node-pjznm\" (UID: \"602e589c-c901-49a2-b9b8-390571f5e5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.526989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526706 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/602e589c-c901-49a2-b9b8-390571f5e5c4-socket-dir\") pod \"aws-ebs-csi-driver-node-pjznm\" (UID: \"602e589c-c901-49a2-b9b8-390571f5e5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.526989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526726 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/602e589c-c901-49a2-b9b8-390571f5e5c4-device-dir\") pod \"aws-ebs-csi-driver-node-pjznm\" (UID: \"602e589c-c901-49a2-b9b8-390571f5e5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.526989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526742 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-multus-socket-dir-parent\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.526989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526766 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-host-run-netns\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.526989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526792 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/56dc0a37-3391-46e8-b21d-71b167fd843f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.526989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526822 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqbtl\" (UniqueName: \"kubernetes.io/projected/602e589c-c901-49a2-b9b8-390571f5e5c4-kube-api-access-vqbtl\") pod \"aws-ebs-csi-driver-node-pjznm\" (UID: \"602e589c-c901-49a2-b9b8-390571f5e5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.526989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526849 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56dc0a37-3391-46e8-b21d-71b167fd843f-cnibin\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.526989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526859 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-host-run-netns\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.526989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526871 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbr98\" (UniqueName: \"kubernetes.io/projected/56dc0a37-3391-46e8-b21d-71b167fd843f-kube-api-access-jbr98\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.526989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526921 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56dc0a37-3391-46e8-b21d-71b167fd843f-cnibin\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.526989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526918 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/602e589c-c901-49a2-b9b8-390571f5e5c4-socket-dir\") pod \"aws-ebs-csi-driver-node-pjznm\" (UID: \"602e589c-c901-49a2-b9b8-390571f5e5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.526989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526958 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4bc5e553-528f-46c0-8724-20c8c6d8024c-agent-certs\") pod \"konnectivity-agent-k24pk\" (UID: \"4bc5e553-528f-46c0-8724-20c8c6d8024c\") " pod="kube-system/konnectivity-agent-k24pk" Jul 14 13:27:20.527870 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.526993 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/602e589c-c901-49a2-b9b8-390571f5e5c4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pjznm\" (UID: \"602e589c-c901-49a2-b9b8-390571f5e5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.527870 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527017 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/602e589c-c901-49a2-b9b8-390571f5e5c4-registration-dir\") pod \"aws-ebs-csi-driver-node-pjznm\" (UID: \"602e589c-c901-49a2-b9b8-390571f5e5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.527870 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527044 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-os-release\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.527870 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527082 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-etc-systemd\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.527870 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527107 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6aec7895-aa6e-40c8-95ec-2b61bf996da0-serviceca\") pod \"node-ca-dqvh8\" (UID: \"6aec7895-aa6e-40c8-95ec-2b61bf996da0\") " pod="openshift-image-registry/node-ca-dqvh8" Jul 14 13:27:20.527870 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527138 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-etc-kubernetes\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.527870 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527188 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-multus-cni-dir\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.527870 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527212 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r52v\" (UniqueName: \"kubernetes.io/projected/86df2bc2-9305-4329-8bc4-53abd12c87ff-kube-api-access-9r52v\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.527870 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527240 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-etc-modprobe-d\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.527870 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527266 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-sys\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.527870 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527296 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-etc-sysconfig\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.527870 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527324 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-host-var-lib-kubelet\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.527870 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527370 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/56dc0a37-3391-46e8-b21d-71b167fd843f-os-release\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.527870 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527398 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-hostroot\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.527870 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527402 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-etc-kubernetes\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.527870 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527453 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/602e589c-c901-49a2-b9b8-390571f5e5c4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pjznm\" (UID: \"602e589c-c901-49a2-b9b8-390571f5e5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.527870 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527468 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-host-run-multus-certs\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.527870 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527488 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/602e589c-c901-49a2-b9b8-390571f5e5c4-registration-dir\") pod \"aws-ebs-csi-driver-node-pjznm\" (UID: \"602e589c-c901-49a2-b9b8-390571f5e5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.528853 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527494 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-lib-modules\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.528853 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527524 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h696j\" (UniqueName: \"kubernetes.io/projected/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-kube-api-access-h696j\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.528853 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527549 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6aec7895-aa6e-40c8-95ec-2b61bf996da0-host\") pod \"node-ca-dqvh8\" (UID: \"6aec7895-aa6e-40c8-95ec-2b61bf996da0\") " pod="openshift-image-registry/node-ca-dqvh8" Jul 14 13:27:20.528853 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527582 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/602e589c-c901-49a2-b9b8-390571f5e5c4-etc-selinux\") pod \"aws-ebs-csi-driver-node-pjznm\" (UID: \"602e589c-c901-49a2-b9b8-390571f5e5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.528853 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527598 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-tmp\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.528853 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527635 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56dc0a37-3391-46e8-b21d-71b167fd843f-system-cni-dir\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.528853 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527643 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-os-release\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.528853 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527670 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-multus-conf-dir\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.528853 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527696 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/86df2bc2-9305-4329-8bc4-53abd12c87ff-multus-daemon-config\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.528853 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527737 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-etc-kubernetes\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.528853 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527755 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-etc-sysctl-d\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.528853 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527765 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-host-run-multus-certs\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.528853 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527771 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-host\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.528853 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527808 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-host\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.528853 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527817 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56dc0a37-3391-46e8-b21d-71b167fd843f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.528853 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527851 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6aec7895-aa6e-40c8-95ec-2b61bf996da0-serviceca\") pod \"node-ca-dqvh8\" (UID: \"6aec7895-aa6e-40c8-95ec-2b61bf996da0\") " pod="openshift-image-registry/node-ca-dqvh8" Jul 14 13:27:20.528853 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527888 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-host-var-lib-cni-multus\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.528853 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527900 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-sys\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.529730 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527911 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-etc-sysconfig\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.529730 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527943 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-run\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.529730 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527945 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-lib-modules\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.529730 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527969 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-host-var-lib-kubelet\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.529730 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527974 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-multus-cni-dir\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.529730 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527973 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-host-var-lib-cni-multus\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.529730 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527542 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-etc-systemd\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.529730 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.528026 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56dc0a37-3391-46e8-b21d-71b167fd843f-cni-binary-copy\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.529730 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.528054 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/56dc0a37-3391-46e8-b21d-71b167fd843f-os-release\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.529730 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.527914 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-run\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.529730 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.528131 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-hostroot\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.529730 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.528147 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-var-lib-kubelet\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.529730 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.528183 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvrjz\" (UniqueName: \"kubernetes.io/projected/6aec7895-aa6e-40c8-95ec-2b61bf996da0-kube-api-access-pvrjz\") pod \"node-ca-dqvh8\" (UID: \"6aec7895-aa6e-40c8-95ec-2b61bf996da0\") " pod="openshift-image-registry/node-ca-dqvh8" Jul 14 13:27:20.529730 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.528215 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/56dc0a37-3391-46e8-b21d-71b167fd843f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.529730 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.528255 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/602e589c-c901-49a2-b9b8-390571f5e5c4-etc-selinux\") pod \"aws-ebs-csi-driver-node-pjznm\" (UID: \"602e589c-c901-49a2-b9b8-390571f5e5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.529730 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.528243 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/86df2bc2-9305-4329-8bc4-53abd12c87ff-cni-binary-copy\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.529730 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.528305 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6aec7895-aa6e-40c8-95ec-2b61bf996da0-host\") pod \"node-ca-dqvh8\" (UID: \"6aec7895-aa6e-40c8-95ec-2b61bf996da0\") " pod="openshift-image-registry/node-ca-dqvh8" Jul 14 13:27:20.529730 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.528320 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-system-cni-dir\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.530516 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.528348 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-etc-tuned\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.530516 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.528376 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4bc5e553-528f-46c0-8724-20c8c6d8024c-konnectivity-ca\") pod \"konnectivity-agent-k24pk\" (UID: \"4bc5e553-528f-46c0-8724-20c8c6d8024c\") " pod="kube-system/konnectivity-agent-k24pk" Jul 14 13:27:20.530516 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.528428 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-system-cni-dir\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.530516 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.528601 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-var-lib-kubelet\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.530516 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.528148 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56dc0a37-3391-46e8-b21d-71b167fd843f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.530516 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.528696 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/86df2bc2-9305-4329-8bc4-53abd12c87ff-multus-conf-dir\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.530516 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.528698 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56dc0a37-3391-46e8-b21d-71b167fd843f-system-cni-dir\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.530516 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.528928 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/56dc0a37-3391-46e8-b21d-71b167fd843f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.530516 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.528993 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4bc5e553-528f-46c0-8724-20c8c6d8024c-konnectivity-ca\") pod \"konnectivity-agent-k24pk\" (UID: \"4bc5e553-528f-46c0-8724-20c8c6d8024c\") " pod="kube-system/konnectivity-agent-k24pk" Jul 14 13:27:20.530516 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.529004 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-etc-kubernetes\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.530516 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.529112 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-etc-modprobe-d\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.530516 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.529126 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-etc-sysctl-d\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.530516 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.529346 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/86df2bc2-9305-4329-8bc4-53abd12c87ff-cni-binary-copy\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.530516 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.529372 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/86df2bc2-9305-4329-8bc4-53abd12c87ff-multus-daemon-config\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.530516 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.529709 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/56dc0a37-3391-46e8-b21d-71b167fd843f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.530516 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.530197 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4bc5e553-528f-46c0-8724-20c8c6d8024c-agent-certs\") pod \"konnectivity-agent-k24pk\" (UID: \"4bc5e553-528f-46c0-8724-20c8c6d8024c\") " pod="kube-system/konnectivity-agent-k24pk" Jul 14 13:27:20.530516 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.530481 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-tmp\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.531324 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.531302 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-etc-tuned\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.542163 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.542102 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbr98\" (UniqueName: \"kubernetes.io/projected/56dc0a37-3391-46e8-b21d-71b167fd843f-kube-api-access-jbr98\") pod \"multus-additional-cni-plugins-xzjv6\" (UID: \"56dc0a37-3391-46e8-b21d-71b167fd843f\") " pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.543230 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.543205 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvrjz\" (UniqueName: \"kubernetes.io/projected/6aec7895-aa6e-40c8-95ec-2b61bf996da0-kube-api-access-pvrjz\") pod \"node-ca-dqvh8\" (UID: \"6aec7895-aa6e-40c8-95ec-2b61bf996da0\") " pod="openshift-image-registry/node-ca-dqvh8" Jul 14 13:27:20.544907 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.544886 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r52v\" (UniqueName: \"kubernetes.io/projected/86df2bc2-9305-4329-8bc4-53abd12c87ff-kube-api-access-9r52v\") pod \"multus-7fmx5\" (UID: \"86df2bc2-9305-4329-8bc4-53abd12c87ff\") " pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.545758 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.545727 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h696j\" (UniqueName: \"kubernetes.io/projected/6e4d8d16-967a-42cd-8d82-7a1d1dcf7115-kube-api-access-h696j\") pod \"tuned-8hmpf\" (UID: \"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115\") " pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.551140 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.551118 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqbtl\" (UniqueName: \"kubernetes.io/projected/602e589c-c901-49a2-b9b8-390571f5e5c4-kube-api-access-vqbtl\") pod \"aws-ebs-csi-driver-node-pjznm\" (UID: \"602e589c-c901-49a2-b9b8-390571f5e5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.610628 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.610592 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-plk87" Jul 14 13:27:20.619768 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.619736 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vjxkn" Jul 14 13:27:20.620905 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:20.620861 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02bc37b4_5dba_4801_92af_07beda478f46.slice/crio-6145f58199507c814ea36efd9a142e52bff90e7e8251aff64642647062e4a7ef WatchSource:0}: Error finding container 6145f58199507c814ea36efd9a142e52bff90e7e8251aff64642647062e4a7ef: Status 404 returned error can't find the container with id 6145f58199507c814ea36efd9a142e52bff90e7e8251aff64642647062e4a7ef Jul 14 13:27:20.628128 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.628102 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:20.629147 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:20.629123 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f52c389_0177_4d3b_93f4_977293414fc1.slice/crio-9dfff886bdfb389cb2140f4e082aa66f6b8ecbbdb47efa2f230f12f5c2db780c WatchSource:0}: Error finding container 9dfff886bdfb389cb2140f4e082aa66f6b8ecbbdb47efa2f230f12f5c2db780c: Status 404 returned error can't find the container with id 9dfff886bdfb389cb2140f4e082aa66f6b8ecbbdb47efa2f230f12f5c2db780c Jul 14 13:27:20.632810 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.632746 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" Jul 14 13:27:20.637448 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.637422 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k24pk" Jul 14 13:27:20.640326 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:20.640298 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod694177f0_32e8_4724_b626_ca46d6c476b5.slice/crio-cf1ddc85e8af9fdb43b98f2f0b058c7e5d9016069d11859aeddf102b045194c7 WatchSource:0}: Error finding container cf1ddc85e8af9fdb43b98f2f0b058c7e5d9016069d11859aeddf102b045194c7: Status 404 returned error can't find the container with id cf1ddc85e8af9fdb43b98f2f0b058c7e5d9016069d11859aeddf102b045194c7 Jul 14 13:27:20.644260 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.644234 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" Jul 14 13:27:20.645049 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:20.645022 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e4d8d16_967a_42cd_8d82_7a1d1dcf7115.slice/crio-769d71ed57466652712fb28236bec690990ecb4bb91df0b72817518d3949cf78 WatchSource:0}: Error finding container 769d71ed57466652712fb28236bec690990ecb4bb91df0b72817518d3949cf78: Status 404 returned error can't find the container with id 769d71ed57466652712fb28236bec690990ecb4bb91df0b72817518d3949cf78 Jul 14 13:27:20.648145 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.648123 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dqvh8" Jul 14 13:27:20.649120 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:20.649094 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bc5e553_528f_46c0_8724_20c8c6d8024c.slice/crio-8835a2edc19130eb69f52015330e998ab2a40dd005b74b7caf919824d4fb16c9 WatchSource:0}: Error finding container 8835a2edc19130eb69f52015330e998ab2a40dd005b74b7caf919824d4fb16c9: Status 404 returned error can't find the container with id 8835a2edc19130eb69f52015330e998ab2a40dd005b74b7caf919824d4fb16c9 Jul 14 13:27:20.653803 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.653780 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7fmx5" Jul 14 13:27:20.655559 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:20.655518 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod602e589c_c901_49a2_b9b8_390571f5e5c4.slice/crio-84172d15d76954534dc042ca486a3bcfc77032fe80841b8d52ea00bae0012c86 WatchSource:0}: Error finding container 84172d15d76954534dc042ca486a3bcfc77032fe80841b8d52ea00bae0012c86: Status 404 returned error can't find the container with id 84172d15d76954534dc042ca486a3bcfc77032fe80841b8d52ea00bae0012c86 Jul 14 13:27:20.658330 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.658311 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xzjv6" Jul 14 13:27:20.658444 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:20.658421 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aec7895_aa6e_40c8_95ec_2b61bf996da0.slice/crio-ef68ece645b627d736c580faa98504a62d5a85e37227884d3e615689e9a916d0 WatchSource:0}: Error finding container ef68ece645b627d736c580faa98504a62d5a85e37227884d3e615689e9a916d0: Status 404 returned error can't find the container with id ef68ece645b627d736c580faa98504a62d5a85e37227884d3e615689e9a916d0 Jul 14 13:27:20.670604 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:20.670579 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56dc0a37_3391_46e8_b21d_71b167fd843f.slice/crio-5d9e2a29d95ad861eb088af68a55b528d9c9ce789f73a41d8d3efce99e8384d8 WatchSource:0}: Error finding container 5d9e2a29d95ad861eb088af68a55b528d9c9ce789f73a41d8d3efce99e8384d8: Status 404 returned error can't find the container with id 5d9e2a29d95ad861eb088af68a55b528d9c9ce789f73a41d8d3efce99e8384d8 Jul 14 13:27:20.931777 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.931670 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret\") pod \"global-pull-secret-syncer-z9kpt\" (UID: \"92645e2c-0c99-4eca-930a-686d53f52229\") " pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:20.931777 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:20.931728 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs\") pod \"network-metrics-daemon-xrgrj\" (UID: \"50a391d6-3c2b-478d-b005-4bbd65bdec74\") " pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:20.932002 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:20.931864 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:20.932002 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:20.931953 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret podName:92645e2c-0c99-4eca-930a-686d53f52229 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:21.931929879 +0000 UTC m=+5.459080514 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret") pod "global-pull-secret-syncer-z9kpt" (UID: "92645e2c-0c99-4eca-930a-686d53f52229") : object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:20.932002 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:20.931873 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:20.932183 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:20.932084 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs podName:50a391d6-3c2b-478d-b005-4bbd65bdec74 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:21.932062735 +0000 UTC m=+5.459213355 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs") pod "network-metrics-daemon-xrgrj" (UID: "50a391d6-3c2b-478d-b005-4bbd65bdec74") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:21.032582 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:21.032541 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cs9c\" (UniqueName: \"kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c\") pod \"network-check-target-wnnf2\" (UID: \"1bf7c74f-a569-44e9-98cb-ee2255c6067e\") " pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:21.032784 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:21.032764 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 13:27:21.032840 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:21.032796 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 13:27:21.032840 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:21.032807 2583 projected.go:196] Error preparing data for projected volume kube-api-access-2cs9c for pod openshift-network-diagnostics/network-check-target-wnnf2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:21.032922 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:21.032874 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c podName:1bf7c74f-a569-44e9-98cb-ee2255c6067e nodeName:}" failed. No retries permitted until 2026-07-14 13:27:22.032853019 +0000 UTC m=+5.560003656 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2cs9c" (UniqueName: "kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c") pod "network-check-target-wnnf2" (UID: "1bf7c74f-a569-44e9-98cb-ee2255c6067e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:21.391442 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:21.391312 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 13:22:20 +0000 UTC" deadline="2028-02-04 17:20:53.51728748 +0000 UTC" Jul 14 13:27:21.391442 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:21.391378 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13683h53m32.125913256s" Jul 14 13:27:21.487834 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:21.487538 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:21.487834 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:21.487694 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnnf2" podUID="1bf7c74f-a569-44e9-98cb-ee2255c6067e" Jul 14 13:27:21.488204 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:21.488167 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:21.488325 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:21.488301 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrgrj" podUID="50a391d6-3c2b-478d-b005-4bbd65bdec74" Jul 14 13:27:21.515805 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:21.513567 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" event={"ID":"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115","Type":"ContainerStarted","Data":"769d71ed57466652712fb28236bec690990ecb4bb91df0b72817518d3949cf78"} Jul 14 13:27:21.519346 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:21.519315 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vjxkn" event={"ID":"2f52c389-0177-4d3b-93f4-977293414fc1","Type":"ContainerStarted","Data":"9dfff886bdfb389cb2140f4e082aa66f6b8ecbbdb47efa2f230f12f5c2db780c"} Jul 14 13:27:21.528540 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:21.528474 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7fmx5" event={"ID":"86df2bc2-9305-4329-8bc4-53abd12c87ff","Type":"ContainerStarted","Data":"99cf1212ba9986d69d3a0bc1da297b0a779814712c5baca86123efafafca7388"} Jul 14 13:27:21.534045 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:21.534012 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" event={"ID":"694177f0-32e8-4724-b626-ca46d6c476b5","Type":"ContainerStarted","Data":"cf1ddc85e8af9fdb43b98f2f0b058c7e5d9016069d11859aeddf102b045194c7"} Jul 14 13:27:21.540447 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:21.540390 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-plk87" event={"ID":"02bc37b4-5dba-4801-92af-07beda478f46","Type":"ContainerStarted","Data":"6145f58199507c814ea36efd9a142e52bff90e7e8251aff64642647062e4a7ef"} Jul 14 13:27:21.549713 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:21.549668 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzjv6" event={"ID":"56dc0a37-3391-46e8-b21d-71b167fd843f","Type":"ContainerStarted","Data":"5d9e2a29d95ad861eb088af68a55b528d9c9ce789f73a41d8d3efce99e8384d8"} Jul 14 13:27:21.552174 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:21.552131 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dqvh8" event={"ID":"6aec7895-aa6e-40c8-95ec-2b61bf996da0","Type":"ContainerStarted","Data":"ef68ece645b627d736c580faa98504a62d5a85e37227884d3e615689e9a916d0"} Jul 14 13:27:21.557741 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:21.556049 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" event={"ID":"602e589c-c901-49a2-b9b8-390571f5e5c4","Type":"ContainerStarted","Data":"84172d15d76954534dc042ca486a3bcfc77032fe80841b8d52ea00bae0012c86"} Jul 14 13:27:21.564259 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:21.564224 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k24pk" event={"ID":"4bc5e553-528f-46c0-8724-20c8c6d8024c","Type":"ContainerStarted","Data":"8835a2edc19130eb69f52015330e998ab2a40dd005b74b7caf919824d4fb16c9"} Jul 14 13:27:21.940775 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:21.940691 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret\") pod \"global-pull-secret-syncer-z9kpt\" (UID: \"92645e2c-0c99-4eca-930a-686d53f52229\") " pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:21.940775 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:21.940750 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs\") pod \"network-metrics-daemon-xrgrj\" (UID: \"50a391d6-3c2b-478d-b005-4bbd65bdec74\") " pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:21.940978 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:21.940899 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:21.940978 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:21.940963 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs podName:50a391d6-3c2b-478d-b005-4bbd65bdec74 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:23.940943114 +0000 UTC m=+7.468093734 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs") pod "network-metrics-daemon-xrgrj" (UID: "50a391d6-3c2b-478d-b005-4bbd65bdec74") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:21.941217 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:21.941041 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:21.941217 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:21.941077 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret podName:92645e2c-0c99-4eca-930a-686d53f52229 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:23.941065771 +0000 UTC m=+7.468216403 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret") pod "global-pull-secret-syncer-z9kpt" (UID: "92645e2c-0c99-4eca-930a-686d53f52229") : object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:22.042136 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:22.041764 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cs9c\" (UniqueName: \"kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c\") pod \"network-check-target-wnnf2\" (UID: \"1bf7c74f-a569-44e9-98cb-ee2255c6067e\") " pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:22.042136 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:22.041951 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 13:27:22.042136 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:22.042125 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 13:27:22.042136 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:22.042146 2583 projected.go:196] Error preparing data for projected volume kube-api-access-2cs9c for pod openshift-network-diagnostics/network-check-target-wnnf2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:22.042441 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:22.042219 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c podName:1bf7c74f-a569-44e9-98cb-ee2255c6067e nodeName:}" failed. No retries permitted until 2026-07-14 13:27:24.042197733 +0000 UTC m=+7.569348349 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2cs9c" (UniqueName: "kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c") pod "network-check-target-wnnf2" (UID: "1bf7c74f-a569-44e9-98cb-ee2255c6067e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:22.392454 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:22.392324 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 13:22:20 +0000 UTC" deadline="2028-01-17 17:28:05.690727526 +0000 UTC" Jul 14 13:27:22.392454 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:22.392366 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13252h0m43.298366954s" Jul 14 13:27:22.489718 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:22.489691 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:22.489890 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:22.489857 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9kpt" podUID="92645e2c-0c99-4eca-930a-686d53f52229" Jul 14 13:27:22.582673 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:22.582631 2583 generic.go:356] "Generic (PLEG): container finished" podID="2adea4ae4de3c3672673fba8673389d5" containerID="650c0511be13edb88c7c9fdcaae054de03f145f9c4a98b852b8da1f3e8313645" exitCode=0 Jul 14 13:27:22.583049 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:22.582699 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-94.ec2.internal" event={"ID":"2adea4ae4de3c3672673fba8673389d5","Type":"ContainerDied","Data":"650c0511be13edb88c7c9fdcaae054de03f145f9c4a98b852b8da1f3e8313645"} Jul 14 13:27:23.486878 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:23.486840 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:23.487397 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:23.486997 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrgrj" podUID="50a391d6-3c2b-478d-b005-4bbd65bdec74" Jul 14 13:27:23.487397 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:23.487269 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:23.487397 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:23.487380 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnnf2" podUID="1bf7c74f-a569-44e9-98cb-ee2255c6067e" Jul 14 13:27:23.604512 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:23.604297 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-94.ec2.internal" event={"ID":"2adea4ae4de3c3672673fba8673389d5","Type":"ContainerStarted","Data":"e875f7cc36459a3c9dfc8cce9035b3c0a0e6086c9f35b4b17acb5fd310461d54"} Jul 14 13:27:23.634960 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:23.634903 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-94.ec2.internal" podStartSLOduration=3.634886721 podStartE2EDuration="3.634886721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:23.634151459 +0000 UTC m=+7.161302098" watchObservedRunningTime="2026-07-14 13:27:23.634886721 +0000 UTC m=+7.162037360" Jul 14 13:27:23.958016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:23.957922 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret\") pod \"global-pull-secret-syncer-z9kpt\" (UID: \"92645e2c-0c99-4eca-930a-686d53f52229\") " pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:23.958016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:23.957980 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs\") pod \"network-metrics-daemon-xrgrj\" (UID: \"50a391d6-3c2b-478d-b005-4bbd65bdec74\") " pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:23.958277 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:23.958167 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:23.958277 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:23.958241 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs podName:50a391d6-3c2b-478d-b005-4bbd65bdec74 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:27.958217916 +0000 UTC m=+11.485368532 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs") pod "network-metrics-daemon-xrgrj" (UID: "50a391d6-3c2b-478d-b005-4bbd65bdec74") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:23.958670 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:23.958647 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:23.958783 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:23.958710 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret podName:92645e2c-0c99-4eca-930a-686d53f52229 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:27.958693002 +0000 UTC m=+11.485843624 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret") pod "global-pull-secret-syncer-z9kpt" (UID: "92645e2c-0c99-4eca-930a-686d53f52229") : object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:24.060067 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:24.059345 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cs9c\" (UniqueName: \"kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c\") pod \"network-check-target-wnnf2\" (UID: \"1bf7c74f-a569-44e9-98cb-ee2255c6067e\") " pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:24.060067 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:24.059592 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 13:27:24.060067 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:24.059614 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 13:27:24.060067 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:24.059629 2583 projected.go:196] Error preparing data for projected volume kube-api-access-2cs9c for pod openshift-network-diagnostics/network-check-target-wnnf2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:24.060067 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:24.059695 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c podName:1bf7c74f-a569-44e9-98cb-ee2255c6067e nodeName:}" failed. No retries permitted until 2026-07-14 13:27:28.059675311 +0000 UTC m=+11.586825934 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2cs9c" (UniqueName: "kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c") pod "network-check-target-wnnf2" (UID: "1bf7c74f-a569-44e9-98cb-ee2255c6067e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:24.488821 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:24.488212 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:24.488821 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:24.488380 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9kpt" podUID="92645e2c-0c99-4eca-930a-686d53f52229" Jul 14 13:27:25.487457 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:25.487419 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:25.487663 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:25.487587 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrgrj" podUID="50a391d6-3c2b-478d-b005-4bbd65bdec74" Jul 14 13:27:25.488012 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:25.487987 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:25.488137 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:25.488094 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnnf2" podUID="1bf7c74f-a569-44e9-98cb-ee2255c6067e" Jul 14 13:27:26.487594 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:26.487557 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:26.488040 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:26.487693 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9kpt" podUID="92645e2c-0c99-4eca-930a-686d53f52229" Jul 14 13:27:27.487689 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:27.487585 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:27.488166 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:27.487732 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:27.488166 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:27.487964 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrgrj" podUID="50a391d6-3c2b-478d-b005-4bbd65bdec74" Jul 14 13:27:27.488166 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:27.487733 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnnf2" podUID="1bf7c74f-a569-44e9-98cb-ee2255c6067e" Jul 14 13:27:27.994569 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:27.994525 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret\") pod \"global-pull-secret-syncer-z9kpt\" (UID: \"92645e2c-0c99-4eca-930a-686d53f52229\") " pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:27.994758 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:27.994588 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs\") pod \"network-metrics-daemon-xrgrj\" (UID: \"50a391d6-3c2b-478d-b005-4bbd65bdec74\") " pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:27.994840 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:27.994773 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:27.994840 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:27.994839 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs podName:50a391d6-3c2b-478d-b005-4bbd65bdec74 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:35.994819163 +0000 UTC m=+19.521969779 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs") pod "network-metrics-daemon-xrgrj" (UID: "50a391d6-3c2b-478d-b005-4bbd65bdec74") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:27.995273 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:27.995252 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:27.995343 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:27.995308 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret podName:92645e2c-0c99-4eca-930a-686d53f52229 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:35.995292122 +0000 UTC m=+19.522442739 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret") pod "global-pull-secret-syncer-z9kpt" (UID: "92645e2c-0c99-4eca-930a-686d53f52229") : object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:28.095381 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:28.095338 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cs9c\" (UniqueName: \"kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c\") pod \"network-check-target-wnnf2\" (UID: \"1bf7c74f-a569-44e9-98cb-ee2255c6067e\") " pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:28.095561 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:28.095485 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 13:27:28.095561 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:28.095502 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 13:27:28.095561 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:28.095514 2583 projected.go:196] Error preparing data for projected volume kube-api-access-2cs9c for pod openshift-network-diagnostics/network-check-target-wnnf2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:28.095715 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:28.095566 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c podName:1bf7c74f-a569-44e9-98cb-ee2255c6067e nodeName:}" failed. No retries permitted until 2026-07-14 13:27:36.095550992 +0000 UTC m=+19.622701625 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2cs9c" (UniqueName: "kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c") pod "network-check-target-wnnf2" (UID: "1bf7c74f-a569-44e9-98cb-ee2255c6067e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:28.488661 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:28.488166 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:28.488661 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:28.488281 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9kpt" podUID="92645e2c-0c99-4eca-930a-686d53f52229" Jul 14 13:27:29.487442 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:29.487229 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:29.487442 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:29.487272 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:29.487442 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:29.487394 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrgrj" podUID="50a391d6-3c2b-478d-b005-4bbd65bdec74" Jul 14 13:27:29.487731 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:29.487518 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnnf2" podUID="1bf7c74f-a569-44e9-98cb-ee2255c6067e" Jul 14 13:27:30.487670 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:30.487261 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:30.487670 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:30.487402 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9kpt" podUID="92645e2c-0c99-4eca-930a-686d53f52229" Jul 14 13:27:31.487341 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:31.487307 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:31.487524 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:31.487306 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:31.487524 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:31.487449 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrgrj" podUID="50a391d6-3c2b-478d-b005-4bbd65bdec74" Jul 14 13:27:31.487524 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:31.487498 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnnf2" podUID="1bf7c74f-a569-44e9-98cb-ee2255c6067e" Jul 14 13:27:32.487370 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:32.487334 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:32.487776 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:32.487477 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9kpt" podUID="92645e2c-0c99-4eca-930a-686d53f52229" Jul 14 13:27:33.487570 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:33.487490 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:33.487570 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:33.487540 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:33.487992 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:33.487618 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrgrj" podUID="50a391d6-3c2b-478d-b005-4bbd65bdec74" Jul 14 13:27:33.487992 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:33.487743 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnnf2" podUID="1bf7c74f-a569-44e9-98cb-ee2255c6067e" Jul 14 13:27:34.486901 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:34.486858 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:34.487074 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:34.487011 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9kpt" podUID="92645e2c-0c99-4eca-930a-686d53f52229" Jul 14 13:27:35.487834 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:35.487794 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:35.488342 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:35.487796 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:35.488342 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:35.487912 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrgrj" podUID="50a391d6-3c2b-478d-b005-4bbd65bdec74" Jul 14 13:27:35.488342 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:35.487991 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnnf2" podUID="1bf7c74f-a569-44e9-98cb-ee2255c6067e" Jul 14 13:27:36.057339 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:36.057261 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret\") pod \"global-pull-secret-syncer-z9kpt\" (UID: \"92645e2c-0c99-4eca-930a-686d53f52229\") " pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:36.057339 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:36.057331 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs\") pod \"network-metrics-daemon-xrgrj\" (UID: \"50a391d6-3c2b-478d-b005-4bbd65bdec74\") " pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:36.057657 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:36.057443 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:36.057657 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:36.057522 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret podName:92645e2c-0c99-4eca-930a-686d53f52229 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:52.05750563 +0000 UTC m=+35.584656248 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret") pod "global-pull-secret-syncer-z9kpt" (UID: "92645e2c-0c99-4eca-930a-686d53f52229") : object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:36.057657 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:36.057453 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:36.057657 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:36.057598 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs podName:50a391d6-3c2b-478d-b005-4bbd65bdec74 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:52.057580403 +0000 UTC m=+35.584731018 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs") pod "network-metrics-daemon-xrgrj" (UID: "50a391d6-3c2b-478d-b005-4bbd65bdec74") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:36.158227 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:36.158181 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cs9c\" (UniqueName: \"kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c\") pod \"network-check-target-wnnf2\" (UID: \"1bf7c74f-a569-44e9-98cb-ee2255c6067e\") " pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:36.158411 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:36.158379 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 13:27:36.158411 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:36.158404 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 13:27:36.158543 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:36.158418 2583 projected.go:196] Error preparing data for projected volume kube-api-access-2cs9c for pod openshift-network-diagnostics/network-check-target-wnnf2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:36.158543 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:36.158493 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c podName:1bf7c74f-a569-44e9-98cb-ee2255c6067e nodeName:}" failed. No retries permitted until 2026-07-14 13:27:52.158464083 +0000 UTC m=+35.685614714 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2cs9c" (UniqueName: "kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c") pod "network-check-target-wnnf2" (UID: "1bf7c74f-a569-44e9-98cb-ee2255c6067e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:36.486848 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:36.486758 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:36.487003 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:36.486891 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9kpt" podUID="92645e2c-0c99-4eca-930a-686d53f52229" Jul 14 13:27:37.487749 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:37.487707 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:37.488246 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:37.487828 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnnf2" podUID="1bf7c74f-a569-44e9-98cb-ee2255c6067e" Jul 14 13:27:37.488246 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:37.487858 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:37.488246 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:37.487937 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrgrj" podUID="50a391d6-3c2b-478d-b005-4bbd65bdec74" Jul 14 13:27:38.488330 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:38.488295 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:38.488770 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:38.488407 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9kpt" podUID="92645e2c-0c99-4eca-930a-686d53f52229" Jul 14 13:27:39.486996 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:39.486961 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:39.487206 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:39.487017 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:39.487206 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:39.487126 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrgrj" podUID="50a391d6-3c2b-478d-b005-4bbd65bdec74" Jul 14 13:27:39.487326 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:39.487271 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnnf2" podUID="1bf7c74f-a569-44e9-98cb-ee2255c6067e" Jul 14 13:27:40.487606 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:40.487574 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:40.488187 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:40.487711 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9kpt" podUID="92645e2c-0c99-4eca-930a-686d53f52229" Jul 14 13:27:41.487250 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.487042 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:41.487419 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.487093 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:41.487419 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:41.487370 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrgrj" podUID="50a391d6-3c2b-478d-b005-4bbd65bdec74" Jul 14 13:27:41.487419 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:41.487396 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnnf2" podUID="1bf7c74f-a569-44e9-98cb-ee2255c6067e" Jul 14 13:27:41.648766 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.648586 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-plk87" event={"ID":"02bc37b4-5dba-4801-92af-07beda478f46","Type":"ContainerStarted","Data":"7bf71008181869f3352b992e12df980edd761fe03d8b36324531027ae2363ab1"} Jul 14 13:27:41.650129 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.650097 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-94.ec2.internal" event={"ID":"e88c1491a47c596d63c06a067a091e22","Type":"ContainerStarted","Data":"0163d173f4798b28bdef1ff6afad01c806587001bf35e3a9f842fd0c82cfa302"} Jul 14 13:27:41.651451 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.651430 2583 generic.go:356] "Generic (PLEG): container finished" podID="56dc0a37-3391-46e8-b21d-71b167fd843f" containerID="0722d518a041af128c7db121d4f6ffa9e580777a161975c4e449ddc5b38f2182" exitCode=0 Jul 14 13:27:41.651534 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.651494 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzjv6" event={"ID":"56dc0a37-3391-46e8-b21d-71b167fd843f","Type":"ContainerDied","Data":"0722d518a041af128c7db121d4f6ffa9e580777a161975c4e449ddc5b38f2182"} Jul 14 13:27:41.653034 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.653014 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dqvh8" event={"ID":"6aec7895-aa6e-40c8-95ec-2b61bf996da0","Type":"ContainerStarted","Data":"7fc6b264b1b499f8ebfa744f13c735374f006df9ccc7393713a44a80b0eb2d37"} Jul 14 13:27:41.654535 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.654516 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" event={"ID":"602e589c-c901-49a2-b9b8-390571f5e5c4","Type":"ContainerStarted","Data":"ed6b2014b3bba0f87a27ea6c923433f6e1ed50bcdc9d3505f0f2da8cc77a31f8"} Jul 14 13:27:41.655816 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.655787 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k24pk" event={"ID":"4bc5e553-528f-46c0-8724-20c8c6d8024c","Type":"ContainerStarted","Data":"6090b14db6e8b844b2a72f7ce83be84f164b3bb414cd9098f357580c8d8844b6"} Jul 14 13:27:41.657296 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.657191 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" event={"ID":"6e4d8d16-967a-42cd-8d82-7a1d1dcf7115","Type":"ContainerStarted","Data":"400fa7c4deb3fd1ec441b75ea3da371672bc70296de1f77b3128a304db265957"} Jul 14 13:27:41.658724 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.658697 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7fmx5" event={"ID":"86df2bc2-9305-4329-8bc4-53abd12c87ff","Type":"ContainerStarted","Data":"5d1593bc5195aec57c34c5b96023ed2e660f0b15b1c57b3601ade6e36019209b"} Jul 14 13:27:41.660904 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.660887 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 13:27:41.661218 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.661200 2583 generic.go:356] "Generic (PLEG): container finished" podID="694177f0-32e8-4724-b626-ca46d6c476b5" containerID="a354b25acc3c3f3400a287a0cfe7ed11e66cbcfef2d7b95fa30dde210fd81419" exitCode=1 Jul 14 13:27:41.661319 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.661235 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" event={"ID":"694177f0-32e8-4724-b626-ca46d6c476b5","Type":"ContainerStarted","Data":"82b23a7ea0941cfeb4d9906d0969d278f8bfa5d49380172620f9eac974ef806f"} Jul 14 13:27:41.661319 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.661279 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" event={"ID":"694177f0-32e8-4724-b626-ca46d6c476b5","Type":"ContainerStarted","Data":"c9ce5707342991f37fc85fb7677b63300fccc532800c17d41520737997adc740"} Jul 14 13:27:41.661388 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.661326 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" event={"ID":"694177f0-32e8-4724-b626-ca46d6c476b5","Type":"ContainerStarted","Data":"73f8bc7ab315b3f4d639e2ee0bc2c42e91a9b96bd6a8a5beaada61c83cc2c1f6"} Jul 14 13:27:41.661388 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.661354 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" event={"ID":"694177f0-32e8-4724-b626-ca46d6c476b5","Type":"ContainerDied","Data":"a354b25acc3c3f3400a287a0cfe7ed11e66cbcfef2d7b95fa30dde210fd81419"} Jul 14 13:27:41.661388 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.661381 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" event={"ID":"694177f0-32e8-4724-b626-ca46d6c476b5","Type":"ContainerStarted","Data":"8d022164dc4411c732d130ca7f4c940d5e13888d91f5745dde2591dea3fc5678"} Jul 14 13:27:41.677686 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.677645 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-plk87" podStartSLOduration=2.32318152 podStartE2EDuration="22.677630603s" totalImagesPullingTime="20.354449083s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:41.67742482 +0000 UTC m=+25.204575459" watchObservedRunningTime="2026-07-14 13:27:41.677630603 +0000 UTC m=+25.204781240" Jul 14 13:27:41.704585 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.704527 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8hmpf" podStartSLOduration=2.368223393 podStartE2EDuration="22.704504529s" totalImagesPullingTime="20.336281136s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:41.704452957 +0000 UTC m=+25.231603592" watchObservedRunningTime="2026-07-14 13:27:41.704504529 +0000 UTC m=+25.231655167" Jul 14 13:27:41.736229 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.736184 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7fmx5" podStartSLOduration=2.364701877 podStartE2EDuration="22.73616641s" totalImagesPullingTime="20.371464533s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:41.735884188 +0000 UTC m=+25.263034823" watchObservedRunningTime="2026-07-14 13:27:41.73616641 +0000 UTC m=+25.263317044" Jul 14 13:27:41.803307 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.803252 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-k24pk" podStartSLOduration=2.476722915 podStartE2EDuration="22.803231032s" totalImagesPullingTime="20.326508117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:27:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:41.8030557 +0000 UTC m=+25.330206334" watchObservedRunningTime="2026-07-14 13:27:41.803231032 +0000 UTC m=+25.330381668" Jul 14 13:27:41.803784 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.803748 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-94.ec2.internal" podStartSLOduration=21.803738986 podStartE2EDuration="21.803738986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:41.770242949 +0000 UTC m=+25.297393586" watchObservedRunningTime="2026-07-14 13:27:41.803738986 +0000 UTC m=+25.330889648" Jul 14 13:27:41.843855 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:41.843808 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dqvh8" podStartSLOduration=2.526879086 podStartE2EDuration="22.843789612s" totalImagesPullingTime="20.316910526s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:41.843645349 +0000 UTC m=+25.370795984" watchObservedRunningTime="2026-07-14 13:27:41.843789612 +0000 UTC m=+25.370940248" Jul 14 13:27:42.487252 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:42.487223 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:42.487392 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:42.487354 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9kpt" podUID="92645e2c-0c99-4eca-930a-686d53f52229" Jul 14 13:27:42.656095 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:42.656058 2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 14 13:27:42.665126 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:42.665095 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" event={"ID":"602e589c-c901-49a2-b9b8-390571f5e5c4","Type":"ContainerStarted","Data":"e47c9b7e88081f7d54c2dffe62eead78aa53737c06e32a0f3254413842d8b1fb"} Jul 14 13:27:42.666644 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:42.666618 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vjxkn" event={"ID":"2f52c389-0177-4d3b-93f4-977293414fc1","Type":"ContainerStarted","Data":"4f0fdd6d0104194994a66d4c93e4dfde3df52fdf7dafdcdfa79b9d42ef5a4ed5"} Jul 14 13:27:42.669732 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:42.669712 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 13:27:42.670958 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:42.670361 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" event={"ID":"694177f0-32e8-4724-b626-ca46d6c476b5","Type":"ContainerStarted","Data":"a88a7b31a8d40b00ede725f12a258ef7c5c3de75002f88eeeecb23dcfa3e60d7"} Jul 14 13:27:42.685396 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:42.685340 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vjxkn" podStartSLOduration=3.340682853 podStartE2EDuration="23.685317813s" totalImagesPullingTime="20.34463496s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:42.684969367 +0000 UTC m=+26.212120002" watchObservedRunningTime="2026-07-14 13:27:42.685317813 +0000 UTC m=+26.212468451" Jul 14 13:27:43.383289 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:43.383192 2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T13:27:42.656094689Z","UUID":"207d42cf-feb3-4710-9266-ddfaeb929741","Handler":null,"Name":"","Endpoint":""} Jul 14 13:27:43.384857 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:43.384835 2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 14 13:27:43.384984 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:43.384882 2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 14 13:27:43.487199 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:43.487165 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:43.487199 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:43.487198 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:43.487416 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:43.487303 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnnf2" podUID="1bf7c74f-a569-44e9-98cb-ee2255c6067e" Jul 14 13:27:43.487487 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:43.487465 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrgrj" podUID="50a391d6-3c2b-478d-b005-4bbd65bdec74" Jul 14 13:27:44.487684 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:44.487483 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:44.488146 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:44.487792 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9kpt" podUID="92645e2c-0c99-4eca-930a-686d53f52229" Jul 14 13:27:44.523890 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:44.523862 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-k24pk" Jul 14 13:27:44.524524 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:44.524481 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-k24pk" Jul 14 13:27:44.540690 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:44.540639 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-k24pk" Jul 14 13:27:44.541227 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:44.541181 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-k24pk" Jul 14 13:27:44.676396 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:44.676360 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" event={"ID":"602e589c-c901-49a2-b9b8-390571f5e5c4","Type":"ContainerStarted","Data":"62561111761bb00ad5f4e0d269bf2935f7764deb01dfe49c4c9fc119975dd174"} Jul 14 13:27:44.679119 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:44.679093 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 13:27:44.679454 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:44.679432 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" event={"ID":"694177f0-32e8-4724-b626-ca46d6c476b5","Type":"ContainerStarted","Data":"b27b42b4323a0bb09bd3efe8b9937cc14fa4f4771cc9196df8fffdb0f8f2fe95"} Jul 14 13:27:45.487244 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:45.487208 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:45.487459 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:45.487248 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:45.487459 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:45.487342 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrgrj" podUID="50a391d6-3c2b-478d-b005-4bbd65bdec74" Jul 14 13:27:45.487539 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:45.487461 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnnf2" podUID="1bf7c74f-a569-44e9-98cb-ee2255c6067e" Jul 14 13:27:46.487111 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:46.486937 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:46.487696 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:46.487198 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9kpt" podUID="92645e2c-0c99-4eca-930a-686d53f52229" Jul 14 13:27:46.685251 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:46.685207 2583 generic.go:356] "Generic (PLEG): container finished" podID="56dc0a37-3391-46e8-b21d-71b167fd843f" containerID="0eb4f0b39ce91a3524ed3d8b0f0ae7c89b8fdd757245ce98a16ab2cd7db17d44" exitCode=0 Jul 14 13:27:46.685426 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:46.685290 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzjv6" event={"ID":"56dc0a37-3391-46e8-b21d-71b167fd843f","Type":"ContainerDied","Data":"0eb4f0b39ce91a3524ed3d8b0f0ae7c89b8fdd757245ce98a16ab2cd7db17d44"} Jul 14 13:27:46.688649 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:46.688625 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 13:27:46.688923 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:46.688905 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" event={"ID":"694177f0-32e8-4724-b626-ca46d6c476b5","Type":"ContainerStarted","Data":"cbb4bca6982fd786649153258639e17d906b9556354303fccf5a8f84e6f9f023"} Jul 14 13:27:46.689121 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:46.689105 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:46.689226 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:46.689175 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:46.689367 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:46.689349 2583 scope.go:117] "RemoveContainer" containerID="a354b25acc3c3f3400a287a0cfe7ed11e66cbcfef2d7b95fa30dde210fd81419" Jul 14 13:27:46.704240 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:46.704223 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:46.712988 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:46.712952 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pjznm" podStartSLOduration=4.186099751 podStartE2EDuration="27.712941049s" totalImagesPullingTime="23.526841298s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:44.696191227 +0000 UTC m=+28.223341863" watchObservedRunningTime="2026-07-14 13:27:46.712941049 +0000 UTC m=+30.240091677" Jul 14 13:27:47.487132 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:47.486792 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:47.487132 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:47.486836 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:47.487132 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:47.486919 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnnf2" podUID="1bf7c74f-a569-44e9-98cb-ee2255c6067e" Jul 14 13:27:47.487751 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:47.487392 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrgrj" podUID="50a391d6-3c2b-478d-b005-4bbd65bdec74" Jul 14 13:27:47.694388 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:47.694362 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 13:27:47.694763 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:47.694734 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" event={"ID":"694177f0-32e8-4724-b626-ca46d6c476b5","Type":"ContainerStarted","Data":"3f95ffcd33bfee63540c32f017d8665b1a421a0b7e661472e89cd40d400f8d9a"} Jul 14 13:27:47.694974 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:47.694954 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:47.709798 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:47.709774 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:27:47.735949 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:47.735900 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" podStartSLOduration=8.330527651 podStartE2EDuration="28.73588563s" totalImagesPullingTime="20.405357979s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:47.73456448 +0000 UTC m=+31.261715115" watchObservedRunningTime="2026-07-14 13:27:47.73588563 +0000 UTC m=+31.263036264" Jul 14 13:27:48.487884 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:48.487853 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:48.488535 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:48.487950 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9kpt" podUID="92645e2c-0c99-4eca-930a-686d53f52229" Jul 14 13:27:48.699214 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:48.699179 2583 generic.go:356] "Generic (PLEG): container finished" podID="56dc0a37-3391-46e8-b21d-71b167fd843f" containerID="ca61b6bc488df3bf680dbf9ddf8ad6e17e17c3eb28b967f39263f4d7654c77e0" exitCode=0 Jul 14 13:27:48.699378 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:48.699274 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzjv6" event={"ID":"56dc0a37-3391-46e8-b21d-71b167fd843f","Type":"ContainerDied","Data":"ca61b6bc488df3bf680dbf9ddf8ad6e17e17c3eb28b967f39263f4d7654c77e0"} Jul 14 13:27:48.742458 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:48.742380 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wnnf2"] Jul 14 13:27:48.742574 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:48.742503 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:48.742616 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:48.742599 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnnf2" podUID="1bf7c74f-a569-44e9-98cb-ee2255c6067e" Jul 14 13:27:48.746303 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:48.746281 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z9kpt"] Jul 14 13:27:48.746426 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:48.746355 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:48.746485 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:48.746421 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9kpt" podUID="92645e2c-0c99-4eca-930a-686d53f52229" Jul 14 13:27:48.750040 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:48.750019 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xrgrj"] Jul 14 13:27:48.750174 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:48.750158 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:48.750318 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:48.750295 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrgrj" podUID="50a391d6-3c2b-478d-b005-4bbd65bdec74" Jul 14 13:27:50.487321 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:50.487290 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:50.487801 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:50.487290 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:50.487801 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:50.487416 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrgrj" podUID="50a391d6-3c2b-478d-b005-4bbd65bdec74" Jul 14 13:27:50.487801 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:50.487430 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:50.487801 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:50.487513 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9kpt" podUID="92645e2c-0c99-4eca-930a-686d53f52229" Jul 14 13:27:50.487801 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:50.487576 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnnf2" podUID="1bf7c74f-a569-44e9-98cb-ee2255c6067e" Jul 14 13:27:50.708686 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:50.708608 2583 generic.go:356] "Generic (PLEG): container finished" podID="56dc0a37-3391-46e8-b21d-71b167fd843f" containerID="dadf7d9cdc863aac2249e00cc568df315813536836d4c3be93fb22746f177228" exitCode=0 Jul 14 13:27:50.708686 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:50.708663 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzjv6" event={"ID":"56dc0a37-3391-46e8-b21d-71b167fd843f","Type":"ContainerDied","Data":"dadf7d9cdc863aac2249e00cc568df315813536836d4c3be93fb22746f177228"} Jul 14 13:27:52.072552 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:52.072517 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret\") pod \"global-pull-secret-syncer-z9kpt\" (UID: \"92645e2c-0c99-4eca-930a-686d53f52229\") " pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:52.072552 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:52.072557 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs\") pod \"network-metrics-daemon-xrgrj\" (UID: \"50a391d6-3c2b-478d-b005-4bbd65bdec74\") " pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:52.072995 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:52.072668 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:52.072995 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:52.072725 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret podName:92645e2c-0c99-4eca-930a-686d53f52229 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:24.072710652 +0000 UTC m=+67.599861269 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret") pod "global-pull-secret-syncer-z9kpt" (UID: "92645e2c-0c99-4eca-930a-686d53f52229") : object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:52.072995 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:52.072668 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:52.072995 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:52.072797 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs podName:50a391d6-3c2b-478d-b005-4bbd65bdec74 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:24.072785258 +0000 UTC m=+67.599935871 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs") pod "network-metrics-daemon-xrgrj" (UID: "50a391d6-3c2b-478d-b005-4bbd65bdec74") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:52.172936 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:52.172900 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cs9c\" (UniqueName: \"kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c\") pod \"network-check-target-wnnf2\" (UID: \"1bf7c74f-a569-44e9-98cb-ee2255c6067e\") " pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:52.173116 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:52.173044 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 13:27:52.173116 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:52.173064 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 13:27:52.173116 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:52.173098 2583 projected.go:196] Error preparing data for projected volume kube-api-access-2cs9c for pod openshift-network-diagnostics/network-check-target-wnnf2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:52.173228 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:52.173154 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c podName:1bf7c74f-a569-44e9-98cb-ee2255c6067e nodeName:}" failed. No retries permitted until 2026-07-14 13:28:24.17313889 +0000 UTC m=+67.700289503 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2cs9c" (UniqueName: "kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c") pod "network-check-target-wnnf2" (UID: "1bf7c74f-a569-44e9-98cb-ee2255c6067e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:52.491265 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:52.491185 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:52.491449 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:52.491293 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:52.491449 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:52.491304 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnnf2" podUID="1bf7c74f-a569-44e9-98cb-ee2255c6067e" Jul 14 13:27:52.491449 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:52.491326 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:52.491449 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:52.491409 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z9kpt" podUID="92645e2c-0c99-4eca-930a-686d53f52229" Jul 14 13:27:52.491631 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:52.491465 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrgrj" podUID="50a391d6-3c2b-478d-b005-4bbd65bdec74" Jul 14 13:27:53.743170 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.743073 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-94.ec2.internal" event="NodeReady" Jul 14 13:27:53.743551 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.743248 2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 14 13:27:53.801435 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.801404 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2zzrk"] Jul 14 13:27:53.803498 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.803463 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2zzrk" Jul 14 13:27:53.804457 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.804431 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2zllr"] Jul 14 13:27:53.806303 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.806279 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2zllr" Jul 14 13:27:53.806574 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.806281 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6rczr\"" Jul 14 13:27:53.806958 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.806910 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 14 13:27:53.806958 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.806946 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 14 13:27:53.809288 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.809148 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-755dc\"" Jul 14 13:27:53.809288 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.809158 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 14 13:27:53.809479 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.809429 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 14 13:27:53.809479 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.809460 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 14 13:27:53.815999 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.815943 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2zzrk"] Jul 14 13:27:53.834709 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.834682 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2zllr"] Jul 14 13:27:53.887268 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.887227 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-metrics-tls\") pod \"dns-default-2zzrk\" (UID: \"82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0\") " pod="openshift-dns/dns-default-2zzrk" Jul 14 13:27:53.887268 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.887270 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8125366b-15c5-4c38-a6ae-25101ed7473c-cert\") pod \"ingress-canary-2zllr\" (UID: \"8125366b-15c5-4c38-a6ae-25101ed7473c\") " pod="openshift-ingress-canary/ingress-canary-2zllr" Jul 14 13:27:53.887492 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.887351 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdmh6\" (UniqueName: \"kubernetes.io/projected/8125366b-15c5-4c38-a6ae-25101ed7473c-kube-api-access-qdmh6\") pod \"ingress-canary-2zllr\" (UID: \"8125366b-15c5-4c38-a6ae-25101ed7473c\") " pod="openshift-ingress-canary/ingress-canary-2zllr" Jul 14 13:27:53.887492 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.887399 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-tmp-dir\") pod \"dns-default-2zzrk\" (UID: \"82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0\") " pod="openshift-dns/dns-default-2zzrk" Jul 14 13:27:53.887492 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.887426 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4m46\" (UniqueName: \"kubernetes.io/projected/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-kube-api-access-m4m46\") pod \"dns-default-2zzrk\" (UID: \"82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0\") " pod="openshift-dns/dns-default-2zzrk" Jul 14 13:27:53.887492 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.887482 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-config-volume\") pod \"dns-default-2zzrk\" (UID: \"82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0\") " pod="openshift-dns/dns-default-2zzrk" Jul 14 13:27:53.988659 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.988627 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-config-volume\") pod \"dns-default-2zzrk\" (UID: \"82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0\") " pod="openshift-dns/dns-default-2zzrk" Jul 14 13:27:53.988831 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.988682 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-metrics-tls\") pod \"dns-default-2zzrk\" (UID: \"82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0\") " pod="openshift-dns/dns-default-2zzrk" Jul 14 13:27:53.988831 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.988713 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8125366b-15c5-4c38-a6ae-25101ed7473c-cert\") pod \"ingress-canary-2zllr\" (UID: \"8125366b-15c5-4c38-a6ae-25101ed7473c\") " pod="openshift-ingress-canary/ingress-canary-2zllr" Jul 14 13:27:53.988831 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.988745 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdmh6\" (UniqueName: \"kubernetes.io/projected/8125366b-15c5-4c38-a6ae-25101ed7473c-kube-api-access-qdmh6\") pod \"ingress-canary-2zllr\" (UID: \"8125366b-15c5-4c38-a6ae-25101ed7473c\") " pod="openshift-ingress-canary/ingress-canary-2zllr" Jul 14 13:27:53.988831 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.988768 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-tmp-dir\") pod \"dns-default-2zzrk\" (UID: \"82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0\") " pod="openshift-dns/dns-default-2zzrk" Jul 14 13:27:53.988831 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.988788 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4m46\" (UniqueName: \"kubernetes.io/projected/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-kube-api-access-m4m46\") pod \"dns-default-2zzrk\" (UID: \"82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0\") " pod="openshift-dns/dns-default-2zzrk" Jul 14 13:27:53.989097 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:53.988875 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 13:27:53.989097 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:53.988948 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-metrics-tls podName:82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:54.488928359 +0000 UTC m=+38.016078976 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-metrics-tls") pod "dns-default-2zzrk" (UID: "82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0") : secret "dns-default-metrics-tls" not found Jul 14 13:27:53.989228 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.989134 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-tmp-dir\") pod \"dns-default-2zzrk\" (UID: \"82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0\") " pod="openshift-dns/dns-default-2zzrk" Jul 14 13:27:53.989390 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.989367 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-config-volume\") pod \"dns-default-2zzrk\" (UID: \"82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0\") " pod="openshift-dns/dns-default-2zzrk" Jul 14 13:27:53.993657 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:53.993601 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8125366b-15c5-4c38-a6ae-25101ed7473c-cert\") pod \"ingress-canary-2zllr\" (UID: \"8125366b-15c5-4c38-a6ae-25101ed7473c\") " pod="openshift-ingress-canary/ingress-canary-2zllr" Jul 14 13:27:54.005661 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:54.005637 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdmh6\" (UniqueName: \"kubernetes.io/projected/8125366b-15c5-4c38-a6ae-25101ed7473c-kube-api-access-qdmh6\") pod \"ingress-canary-2zllr\" (UID: \"8125366b-15c5-4c38-a6ae-25101ed7473c\") " pod="openshift-ingress-canary/ingress-canary-2zllr" Jul 14 13:27:54.006361 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:54.006338 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4m46\" (UniqueName: \"kubernetes.io/projected/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-kube-api-access-m4m46\") pod \"dns-default-2zzrk\" (UID: \"82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0\") " pod="openshift-dns/dns-default-2zzrk" Jul 14 13:27:54.121766 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:54.121728 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2zllr" Jul 14 13:27:54.267307 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:54.267115 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2zllr"] Jul 14 13:27:54.487615 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:54.487578 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:27:54.487786 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:54.487581 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:27:54.487786 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:54.487692 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:27:54.490964 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:54.490911 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 13:27:54.492388 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:54.492370 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l7sfg\"" Jul 14 13:27:54.492474 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:54.492405 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 13:27:54.492474 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:54.492419 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-metrics-tls\") pod \"dns-default-2zzrk\" (UID: \"82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0\") " pod="openshift-dns/dns-default-2zzrk" Jul 14 13:27:54.492539 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:54.492529 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rgn8m\"" Jul 14 13:27:54.492615 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:54.492600 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 13:27:54.492657 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:54.492621 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 13:27:54.492686 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:54.492668 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-metrics-tls podName:82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:55.492655895 +0000 UTC m=+39.019806509 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-metrics-tls") pod "dns-default-2zzrk" (UID: "82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0") : secret "dns-default-metrics-tls" not found Jul 14 13:27:54.492769 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:54.492753 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 13:27:54.788731 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:54.788689 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:27:54.884287 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:54.884252 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:27:55.500734 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:55.500699 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-metrics-tls\") pod \"dns-default-2zzrk\" (UID: \"82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0\") " pod="openshift-dns/dns-default-2zzrk" Jul 14 13:27:55.500913 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:55.500835 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 13:27:55.500913 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:55.500904 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-metrics-tls podName:82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:57.500888048 +0000 UTC m=+41.028038661 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-metrics-tls") pod "dns-default-2zzrk" (UID: "82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0") : secret "dns-default-metrics-tls" not found Jul 14 13:27:55.535729 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:55.535692 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:27:55.633051 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:55.633016 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:27:55.663810 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:27:55.663784 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8125366b_15c5_4c38_a6ae_25101ed7473c.slice/crio-4a9482b336398cc268a21ceb21151225518c280076d3ecf35ba6be79e17a270e WatchSource:0}: Error finding container 4a9482b336398cc268a21ceb21151225518c280076d3ecf35ba6be79e17a270e: Status 404 returned error can't find the container with id 4a9482b336398cc268a21ceb21151225518c280076d3ecf35ba6be79e17a270e Jul 14 13:27:55.722269 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:55.722240 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2zllr" event={"ID":"8125366b-15c5-4c38-a6ae-25101ed7473c","Type":"ContainerStarted","Data":"4a9482b336398cc268a21ceb21151225518c280076d3ecf35ba6be79e17a270e"} Jul 14 13:27:56.727956 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:56.727919 2583 generic.go:356] "Generic (PLEG): container finished" podID="56dc0a37-3391-46e8-b21d-71b167fd843f" containerID="1ed9a9c2bddc3c7f6ce44806ae60033e1ad6fa4dcc1f56110e5e6df3610391bc" exitCode=0 Jul 14 13:27:56.728458 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:56.727957 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzjv6" event={"ID":"56dc0a37-3391-46e8-b21d-71b167fd843f","Type":"ContainerDied","Data":"1ed9a9c2bddc3c7f6ce44806ae60033e1ad6fa4dcc1f56110e5e6df3610391bc"} Jul 14 13:27:57.516834 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:57.516787 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-metrics-tls\") pod \"dns-default-2zzrk\" (UID: \"82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0\") " pod="openshift-dns/dns-default-2zzrk" Jul 14 13:27:57.517031 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:57.517008 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 13:27:57.517128 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:27:57.517114 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-metrics-tls podName:82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:01.517063988 +0000 UTC m=+45.044214615 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-metrics-tls") pod "dns-default-2zzrk" (UID: "82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0") : secret "dns-default-metrics-tls" not found Jul 14 13:27:57.734608 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:57.734563 2583 generic.go:356] "Generic (PLEG): container finished" podID="56dc0a37-3391-46e8-b21d-71b167fd843f" containerID="02c2eb0aff6c0b4edfe061d028b3968f4d453c2f3aabb4de20ec86cba3fe7539" exitCode=0 Jul 14 13:27:57.734976 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:57.734615 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzjv6" event={"ID":"56dc0a37-3391-46e8-b21d-71b167fd843f","Type":"ContainerDied","Data":"02c2eb0aff6c0b4edfe061d028b3968f4d453c2f3aabb4de20ec86cba3fe7539"} Jul 14 13:27:57.919636 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:57.919553 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:27:58.023045 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:58.022845 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:27:58.738242 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:58.738203 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2zllr" event={"ID":"8125366b-15c5-4c38-a6ae-25101ed7473c","Type":"ContainerStarted","Data":"2a9b518b174ad2c18767f0639f382c9dd942c3339f4632e3ee607fdc77e6665f"} Jul 14 13:27:58.741474 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:58.741452 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzjv6" event={"ID":"56dc0a37-3391-46e8-b21d-71b167fd843f","Type":"ContainerStarted","Data":"6b4171fc51b8a8c0ddf4e16d7d3b59fa99cdbf6462d3141f87ef60c5b68c6cb6"} Jul 14 13:27:58.781371 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:58.780802 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xzjv6" podStartSLOduration=7.255279623 podStartE2EDuration="39.780781487s" totalImagesPullingTime="32.525501864s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:58.778919019 +0000 UTC m=+42.306069663" watchObservedRunningTime="2026-07-14 13:27:58.780781487 +0000 UTC m=+42.307932122" Jul 14 13:27:58.781371 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:58.780959 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2zllr" podStartSLOduration=3.334970249 podStartE2EDuration="5.780949508s" totalImagesPullingTime="2.445979259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:58.755060841 +0000 UTC m=+42.282211476" watchObservedRunningTime="2026-07-14 13:27:58.780949508 +0000 UTC m=+42.308100145" Jul 14 13:27:58.935361 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:58.935321 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:27:59.033317 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:59.033252 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:27:59.344827 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:59.344752 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:27:59.420805 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:59.420768 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:27:59.609593 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:59.609507 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:27:59.683648 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:59.683613 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:27:59.752807 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:27:59.752770 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:00.280497 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.280460 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:00.300908 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.300878 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b68dc"] Jul 14 13:28:00.333345 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.333312 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b68dc"] Jul 14 13:28:00.333512 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.333453 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b68dc" Jul 14 13:28:00.336330 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.336309 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 14 13:28:00.336424 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.336311 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 14 13:28:00.336424 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.336359 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-4d5n7\"" Jul 14 13:28:00.356583 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.356547 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:00.424973 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.424939 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:00.441770 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.441739 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g9v9\" (UniqueName: \"kubernetes.io/projected/309f8182-1eb0-4ef2-8ff4-b33a08bde279-kube-api-access-7g9v9\") pod \"migrator-7f59b54f6c-b68dc\" (UID: \"309f8182-1eb0-4ef2-8ff4-b33a08bde279\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b68dc" Jul 14 13:28:00.543164 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.543056 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g9v9\" (UniqueName: \"kubernetes.io/projected/309f8182-1eb0-4ef2-8ff4-b33a08bde279-kube-api-access-7g9v9\") pod \"migrator-7f59b54f6c-b68dc\" (UID: \"309f8182-1eb0-4ef2-8ff4-b33a08bde279\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b68dc" Jul 14 13:28:00.550134 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.550106 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jmjw8"] Jul 14 13:28:00.562544 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.562519 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g9v9\" (UniqueName: \"kubernetes.io/projected/309f8182-1eb0-4ef2-8ff4-b33a08bde279-kube-api-access-7g9v9\") pod \"migrator-7f59b54f6c-b68dc\" (UID: \"309f8182-1eb0-4ef2-8ff4-b33a08bde279\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b68dc" Jul 14 13:28:00.569447 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.569420 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:00.571630 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.571612 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jmjw8"] Jul 14 13:28:00.571736 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.571721 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:00.574719 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.574698 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 14 13:28:00.574850 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.574752 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tbwpd\"" Jul 14 13:28:00.574850 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.574753 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 14 13:28:00.575136 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.575114 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 14 13:28:00.575239 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.575223 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 14 13:28:00.642234 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.642200 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b68dc" Jul 14 13:28:00.744137 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.744104 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-data-volume\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:00.744286 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.744158 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jjlv\" (UniqueName: \"kubernetes.io/projected/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-kube-api-access-9jjlv\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:00.744286 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.744217 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:00.744378 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.744288 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-crio-socket\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:00.744378 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.744345 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:00.785115 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.785068 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b68dc"] Jul 14 13:28:00.788405 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:28:00.788375 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod309f8182_1eb0_4ef2_8ff4_b33a08bde279.slice/crio-2888eaac306af3babc47cbd5eab6a7c11722515ce9634ecf994a8a7145c9ac29 WatchSource:0}: Error finding container 2888eaac306af3babc47cbd5eab6a7c11722515ce9634ecf994a8a7145c9ac29: Status 404 returned error can't find the container with id 2888eaac306af3babc47cbd5eab6a7c11722515ce9634ecf994a8a7145c9ac29 Jul 14 13:28:00.805715 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.805630 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:00.844766 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.844732 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-data-volume\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:00.844766 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.844769 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jjlv\" (UniqueName: \"kubernetes.io/projected/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-kube-api-access-9jjlv\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:00.844923 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.844801 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:00.844923 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:00.844872 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 13:28:00.844988 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.844923 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-crio-socket\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:00.844988 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:00.844939 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-kube-rbac-proxy-cm podName:6f321dd9-1f3d-42c0-b6eb-05130c73d8b9 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:01.344917372 +0000 UTC m=+44.872068000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jmjw8" (UID: "6f321dd9-1f3d-42c0-b6eb-05130c73d8b9") : configmap "kube-rbac-proxy" not found Jul 14 13:28:00.844988 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.844963 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:00.845130 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:00.845048 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 13:28:00.845130 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:00.845102 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-insights-runtime-extractor-tls podName:6f321dd9-1f3d-42c0-b6eb-05130c73d8b9 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:01.345069873 +0000 UTC m=+44.872220501 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jmjw8" (UID: "6f321dd9-1f3d-42c0-b6eb-05130c73d8b9") : secret "insights-runtime-extractor-tls" not found Jul 14 13:28:00.845196 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.845126 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-crio-socket\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:00.845196 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.845154 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-data-volume\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:00.860015 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.859988 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jjlv\" (UniqueName: \"kubernetes.io/projected/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-kube-api-access-9jjlv\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:00.880856 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:00.880823 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:01.017563 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:01.017533 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:01.107837 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:01.107760 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:01.349122 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:01.349065 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:01.349122 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:01.349126 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:01.349326 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:01.349186 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 13:28:01.349326 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:01.349256 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-kube-rbac-proxy-cm podName:6f321dd9-1f3d-42c0-b6eb-05130c73d8b9 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:02.34923871 +0000 UTC m=+45.876389323 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jmjw8" (UID: "6f321dd9-1f3d-42c0-b6eb-05130c73d8b9") : configmap "kube-rbac-proxy" not found Jul 14 13:28:01.349326 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:01.349256 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 13:28:01.349326 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:01.349300 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-insights-runtime-extractor-tls podName:6f321dd9-1f3d-42c0-b6eb-05130c73d8b9 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:02.349289131 +0000 UTC m=+45.876439745 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jmjw8" (UID: "6f321dd9-1f3d-42c0-b6eb-05130c73d8b9") : secret "insights-runtime-extractor-tls" not found Jul 14 13:28:01.517174 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:01.517141 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:01.551027 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:01.550999 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-metrics-tls\") pod \"dns-default-2zzrk\" (UID: \"82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0\") " pod="openshift-dns/dns-default-2zzrk" Jul 14 13:28:01.551185 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:01.551166 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 13:28:01.551242 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:01.551233 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-metrics-tls podName:82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:09.551216602 +0000 UTC m=+53.078367215 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-metrics-tls") pod "dns-default-2zzrk" (UID: "82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0") : secret "dns-default-metrics-tls" not found Jul 14 13:28:01.607953 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:01.607906 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:01.749952 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:01.749916 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b68dc" event={"ID":"309f8182-1eb0-4ef2-8ff4-b33a08bde279","Type":"ContainerStarted","Data":"2888eaac306af3babc47cbd5eab6a7c11722515ce9634ecf994a8a7145c9ac29"} Jul 14 13:28:02.358341 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:02.358294 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:02.358787 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:02.358442 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:02.358787 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:02.358478 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 13:28:02.358787 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:02.358531 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 13:28:02.358787 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:02.358561 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-insights-runtime-extractor-tls podName:6f321dd9-1f3d-42c0-b6eb-05130c73d8b9 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:04.358540847 +0000 UTC m=+47.885691464 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jmjw8" (UID: "6f321dd9-1f3d-42c0-b6eb-05130c73d8b9") : secret "insights-runtime-extractor-tls" not found Jul 14 13:28:02.358787 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:02.358581 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-kube-rbac-proxy-cm podName:6f321dd9-1f3d-42c0-b6eb-05130c73d8b9 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:04.358571552 +0000 UTC m=+47.885722170 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jmjw8" (UID: "6f321dd9-1f3d-42c0-b6eb-05130c73d8b9") : configmap "kube-rbac-proxy" not found Jul 14 13:28:03.107101 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.105706 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-dz7w7"] Jul 14 13:28:03.124055 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.124032 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-dz7w7" Jul 14 13:28:03.129369 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.129343 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-dz7w7"] Jul 14 13:28:03.133586 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.133566 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 14 13:28:03.133701 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.133589 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 14 13:28:03.133998 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.133981 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-2w4qv\"" Jul 14 13:28:03.134094 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.133984 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 14 13:28:03.134162 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.134105 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 14 13:28:03.250176 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.250133 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:03.265350 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.265326 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clb77\" (UniqueName: \"kubernetes.io/projected/e8dc654f-7f81-4e8f-b8d6-1279e1342393-kube-api-access-clb77\") pod \"service-ca-79f68b9f4f-dz7w7\" (UID: \"e8dc654f-7f81-4e8f-b8d6-1279e1342393\") " pod="openshift-service-ca/service-ca-79f68b9f4f-dz7w7" Jul 14 13:28:03.265425 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.265373 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e8dc654f-7f81-4e8f-b8d6-1279e1342393-signing-key\") pod \"service-ca-79f68b9f4f-dz7w7\" (UID: \"e8dc654f-7f81-4e8f-b8d6-1279e1342393\") " pod="openshift-service-ca/service-ca-79f68b9f4f-dz7w7" Jul 14 13:28:03.265458 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.265448 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e8dc654f-7f81-4e8f-b8d6-1279e1342393-signing-cabundle\") pod \"service-ca-79f68b9f4f-dz7w7\" (UID: \"e8dc654f-7f81-4e8f-b8d6-1279e1342393\") " pod="openshift-service-ca/service-ca-79f68b9f4f-dz7w7" Jul 14 13:28:03.365869 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.365781 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e8dc654f-7f81-4e8f-b8d6-1279e1342393-signing-key\") pod \"service-ca-79f68b9f4f-dz7w7\" (UID: \"e8dc654f-7f81-4e8f-b8d6-1279e1342393\") " pod="openshift-service-ca/service-ca-79f68b9f4f-dz7w7" Jul 14 13:28:03.365869 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.365857 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e8dc654f-7f81-4e8f-b8d6-1279e1342393-signing-cabundle\") pod \"service-ca-79f68b9f4f-dz7w7\" (UID: \"e8dc654f-7f81-4e8f-b8d6-1279e1342393\") " pod="openshift-service-ca/service-ca-79f68b9f4f-dz7w7" Jul 14 13:28:03.366367 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.365905 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clb77\" (UniqueName: \"kubernetes.io/projected/e8dc654f-7f81-4e8f-b8d6-1279e1342393-kube-api-access-clb77\") pod \"service-ca-79f68b9f4f-dz7w7\" (UID: \"e8dc654f-7f81-4e8f-b8d6-1279e1342393\") " pod="openshift-service-ca/service-ca-79f68b9f4f-dz7w7" Jul 14 13:28:03.366570 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.366549 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e8dc654f-7f81-4e8f-b8d6-1279e1342393-signing-cabundle\") pod \"service-ca-79f68b9f4f-dz7w7\" (UID: \"e8dc654f-7f81-4e8f-b8d6-1279e1342393\") " pod="openshift-service-ca/service-ca-79f68b9f4f-dz7w7" Jul 14 13:28:03.368349 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.368326 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e8dc654f-7f81-4e8f-b8d6-1279e1342393-signing-key\") pod \"service-ca-79f68b9f4f-dz7w7\" (UID: \"e8dc654f-7f81-4e8f-b8d6-1279e1342393\") " pod="openshift-service-ca/service-ca-79f68b9f4f-dz7w7" Jul 14 13:28:03.376804 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.376781 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clb77\" (UniqueName: \"kubernetes.io/projected/e8dc654f-7f81-4e8f-b8d6-1279e1342393-kube-api-access-clb77\") pod \"service-ca-79f68b9f4f-dz7w7\" (UID: \"e8dc654f-7f81-4e8f-b8d6-1279e1342393\") " pod="openshift-service-ca/service-ca-79f68b9f4f-dz7w7" Jul 14 13:28:03.433832 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.433793 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-dz7w7" Jul 14 13:28:03.522760 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.522733 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-plk87_02bc37b4-5dba-4801-92af-07beda478f46/dns-node-resolver/0.log" Jul 14 13:28:03.578840 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.578808 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-dz7w7"] Jul 14 13:28:03.581804 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:28:03.581775 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8dc654f_7f81_4e8f_b8d6_1279e1342393.slice/crio-bdabc727abfa97a0e4f9ab9e7fce000b8e82f029bce949d06707450b067b303d WatchSource:0}: Error finding container bdabc727abfa97a0e4f9ab9e7fce000b8e82f029bce949d06707450b067b303d: Status 404 returned error can't find the container with id bdabc727abfa97a0e4f9ab9e7fce000b8e82f029bce949d06707450b067b303d Jul 14 13:28:03.756498 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.756467 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b68dc" event={"ID":"309f8182-1eb0-4ef2-8ff4-b33a08bde279","Type":"ContainerStarted","Data":"f10ab5b991645793161c75382261361a60634641c22d002def58df1d7f65516f"} Jul 14 13:28:03.756654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.756525 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b68dc" event={"ID":"309f8182-1eb0-4ef2-8ff4-b33a08bde279","Type":"ContainerStarted","Data":"79e0072187ed5c5aa0fade6b4928bbf9fd298e19f180a1fe43ecedbb2443875d"} Jul 14 13:28:03.757560 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.757536 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-dz7w7" event={"ID":"e8dc654f-7f81-4e8f-b8d6-1279e1342393","Type":"ContainerStarted","Data":"bdabc727abfa97a0e4f9ab9e7fce000b8e82f029bce949d06707450b067b303d"} Jul 14 13:28:03.777990 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.777947 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b68dc" podStartSLOduration=1.769559208 podStartE2EDuration="3.777932771s" totalImagesPullingTime="2.008373563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:28:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:03.777735656 +0000 UTC m=+47.304886292" watchObservedRunningTime="2026-07-14 13:28:03.777932771 +0000 UTC m=+47.305083407" Jul 14 13:28:03.881575 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.881542 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:03.951731 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:03.951691 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:04.321196 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:04.321170 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dqvh8_6aec7895-aa6e-40c8-95ec-2b61bf996da0/node-ca/0.log" Jul 14 13:28:04.373476 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:04.373422 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:04.373887 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:04.373485 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:04.373887 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:04.373561 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 13:28:04.373887 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:04.373612 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 13:28:04.373887 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:04.373633 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-kube-rbac-proxy-cm podName:6f321dd9-1f3d-42c0-b6eb-05130c73d8b9 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:08.373616028 +0000 UTC m=+51.900766665 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jmjw8" (UID: "6f321dd9-1f3d-42c0-b6eb-05130c73d8b9") : configmap "kube-rbac-proxy" not found Jul 14 13:28:04.373887 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:04.373666 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-insights-runtime-extractor-tls podName:6f321dd9-1f3d-42c0-b6eb-05130c73d8b9 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:08.373651554 +0000 UTC m=+51.900802170 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jmjw8" (UID: "6f321dd9-1f3d-42c0-b6eb-05130c73d8b9") : secret "insights-runtime-extractor-tls" not found Jul 14 13:28:04.451598 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:04.451565 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:04.718625 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:04.718546 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:04.921723 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:04.921676 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2zllr_8125366b-15c5-4c38-a6ae-25101ed7473c/serve-healthcheck-canary/0.log" Jul 14 13:28:05.528895 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:05.528862 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-b68dc_309f8182-1eb0-4ef2-8ff4-b33a08bde279/migrator/0.log" Jul 14 13:28:05.721617 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:05.721586 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-b68dc_309f8182-1eb0-4ef2-8ff4-b33a08bde279/graceful-termination/0.log" Jul 14 13:28:06.766875 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:06.766845 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-dz7w7" event={"ID":"e8dc654f-7f81-4e8f-b8d6-1279e1342393","Type":"ContainerStarted","Data":"0b8795c6a68775f2f74a1f543783b5600ef16e25fa5e6cff748ff49e9dd0b23e"} Jul 14 13:28:07.708454 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:07.708421 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:07.929287 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:07.929233 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:08.030905 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:08.030866 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:08.139283 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:08.139238 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:08.405595 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:08.405515 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:08.405595 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:08.405553 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:08.405817 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:08.405647 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 13:28:08.405817 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:08.405720 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-kube-rbac-proxy-cm podName:6f321dd9-1f3d-42c0-b6eb-05130c73d8b9 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:16.405700118 +0000 UTC m=+59.932850735 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jmjw8" (UID: "6f321dd9-1f3d-42c0-b6eb-05130c73d8b9") : configmap "kube-rbac-proxy" not found Jul 14 13:28:08.405817 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:08.405648 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 13:28:08.405817 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:08.405775 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-insights-runtime-extractor-tls podName:6f321dd9-1f3d-42c0-b6eb-05130c73d8b9 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:16.405764146 +0000 UTC m=+59.932914760 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jmjw8" (UID: "6f321dd9-1f3d-42c0-b6eb-05130c73d8b9") : secret "insights-runtime-extractor-tls" not found Jul 14 13:28:09.006773 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:09.006735 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:09.080297 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:09.080264 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:09.408569 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:09.408473 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:09.611294 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:09.611260 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:09.614124 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:09.614074 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-metrics-tls\") pod \"dns-default-2zzrk\" (UID: \"82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0\") " pod="openshift-dns/dns-default-2zzrk" Jul 14 13:28:09.614270 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:09.614194 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 13:28:09.614270 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:09.614255 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-metrics-tls podName:82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:25.614241565 +0000 UTC m=+69.141392179 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-metrics-tls") pod "dns-default-2zzrk" (UID: "82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0") : secret "dns-default-metrics-tls" not found Jul 14 13:28:09.811057 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:09.811024 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:10.005285 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:10.005226 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:11.002292 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:11.002258 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:11.383695 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:11.383605 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:11.449593 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:11.449558 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:11.600656 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:11.600616 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:11.801234 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:11.801182 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:12.235649 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:12.235565 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:12.313845 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:12.313807 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:12.816471 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:12.816437 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:13.208328 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:13.208237 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:13.657025 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:13.656987 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:13.936267 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:13.936174 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:14.041455 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:14.041413 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:14.409917 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:14.409880 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:14.600962 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:14.600928 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:14.999112 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:14.999058 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:15.065419 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:15.065387 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:15.411600 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:15.411514 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:16.205946 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:16.205913 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:16.342066 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:16.342029 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:16.463735 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:16.463625 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:16.463735 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:16.463675 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:16.463957 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:16.463811 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 13:28:16.463957 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:28:16.463865 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-insights-runtime-extractor-tls podName:6f321dd9-1f3d-42c0-b6eb-05130c73d8b9 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:32.463851731 +0000 UTC m=+75.991002346 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jmjw8" (UID: "6f321dd9-1f3d-42c0-b6eb-05130c73d8b9") : secret "insights-runtime-extractor-tls" not found Jul 14 13:28:16.464339 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:16.464314 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:16.597975 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:16.597936 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:16.804972 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:16.804929 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:17.001829 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:17.001794 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:17.402648 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:17.402616 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:17.996703 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:17.996662 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:18.206494 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:18.206458 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:18.411866 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:18.411753 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:19.719475 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:19.719441 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lpnff" Jul 14 13:28:19.755184 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:19.755132 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-dz7w7" podStartSLOduration=13.97811199 podStartE2EDuration="16.755116757s" totalImagesPullingTime="2.777004767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:06.796793147 +0000 UTC m=+50.323943782" watchObservedRunningTime="2026-07-14 13:28:19.755116757 +0000 UTC m=+63.282267429" Jul 14 13:28:20.730192 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:20.730155 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:20.837635 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:20.837600 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:21.035708 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:21.035674 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:21.406410 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:21.406314 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:21.604468 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:21.604428 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:22.041214 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:22.041174 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:22.113457 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:22.113424 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:22.207211 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:22.207178 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:22.307962 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:22.307870 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:22.426857 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:22.426821 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:22.691663 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:22.691569 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:23.834623 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:23.834590 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:23.901954 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:23.901919 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:24.117799 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:24.117690 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret\") pod \"global-pull-secret-syncer-z9kpt\" (UID: \"92645e2c-0c99-4eca-930a-686d53f52229\") " pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:28:24.117799 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:24.117745 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs\") pod \"network-metrics-daemon-xrgrj\" (UID: \"50a391d6-3c2b-478d-b005-4bbd65bdec74\") " pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:28:24.121008 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:24.120988 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 13:28:24.121067 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:24.121006 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 13:28:24.130486 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:24.130451 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92645e2c-0c99-4eca-930a-686d53f52229-original-pull-secret\") pod \"global-pull-secret-syncer-z9kpt\" (UID: \"92645e2c-0c99-4eca-930a-686d53f52229\") " pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:28:24.130619 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:24.130546 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50a391d6-3c2b-478d-b005-4bbd65bdec74-metrics-certs\") pod \"network-metrics-daemon-xrgrj\" (UID: \"50a391d6-3c2b-478d-b005-4bbd65bdec74\") " pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:28:24.199668 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:24.199617 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z9kpt" Jul 14 13:28:24.214891 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:24.214863 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l7sfg\"" Jul 14 13:28:24.218421 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:24.218402 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cs9c\" (UniqueName: \"kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c\") pod \"network-check-target-wnnf2\" (UID: \"1bf7c74f-a569-44e9-98cb-ee2255c6067e\") " pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:28:24.221294 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:24.221269 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 13:28:24.222217 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:24.222197 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrgrj" Jul 14 13:28:24.233152 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:24.232166 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 13:28:24.242868 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:24.242840 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cs9c\" (UniqueName: \"kubernetes.io/projected/1bf7c74f-a569-44e9-98cb-ee2255c6067e-kube-api-access-2cs9c\") pod \"network-check-target-wnnf2\" (UID: \"1bf7c74f-a569-44e9-98cb-ee2255c6067e\") " pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:28:24.334718 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:24.334654 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z9kpt"] Jul 14 13:28:24.338818 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:28:24.338787 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92645e2c_0c99_4eca_930a_686d53f52229.slice/crio-2efad5340c72d2cdf95c5a47817648fca3cdcbcb98b68c7083a5b39e454938fe WatchSource:0}: Error finding container 2efad5340c72d2cdf95c5a47817648fca3cdcbcb98b68c7083a5b39e454938fe: Status 404 returned error can't find the container with id 2efad5340c72d2cdf95c5a47817648fca3cdcbcb98b68c7083a5b39e454938fe Jul 14 13:28:24.354918 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:24.354891 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xrgrj"] Jul 14 13:28:24.357485 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:28:24.357458 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50a391d6_3c2b_478d_b005_4bbd65bdec74.slice/crio-0deb25dc310daecb21c59f805047c2496938f99ed42b15b96294758b00082a6c WatchSource:0}: Error finding container 0deb25dc310daecb21c59f805047c2496938f99ed42b15b96294758b00082a6c: Status 404 returned error can't find the container with id 0deb25dc310daecb21c59f805047c2496938f99ed42b15b96294758b00082a6c Jul 14 13:28:24.509653 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:24.509628 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rgn8m\"" Jul 14 13:28:24.517473 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:24.517444 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:28:24.643475 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:24.643440 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wnnf2"] Jul 14 13:28:24.646455 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:28:24.646436 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bf7c74f_a569_44e9_98cb_ee2255c6067e.slice/crio-d450349c49d9b99d64098bcc85df0982ae28eb3bb7c197d8833d63aface68b76 WatchSource:0}: Error finding container d450349c49d9b99d64098bcc85df0982ae28eb3bb7c197d8833d63aface68b76: Status 404 returned error can't find the container with id d450349c49d9b99d64098bcc85df0982ae28eb3bb7c197d8833d63aface68b76 Jul 14 13:28:24.813552 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:24.813461 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xrgrj" event={"ID":"50a391d6-3c2b-478d-b005-4bbd65bdec74","Type":"ContainerStarted","Data":"0deb25dc310daecb21c59f805047c2496938f99ed42b15b96294758b00082a6c"} Jul 14 13:28:24.814857 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:24.814828 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wnnf2" event={"ID":"1bf7c74f-a569-44e9-98cb-ee2255c6067e","Type":"ContainerStarted","Data":"d450349c49d9b99d64098bcc85df0982ae28eb3bb7c197d8833d63aface68b76"} Jul 14 13:28:24.816122 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:24.816092 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z9kpt" event={"ID":"92645e2c-0c99-4eca-930a-686d53f52229","Type":"ContainerStarted","Data":"2efad5340c72d2cdf95c5a47817648fca3cdcbcb98b68c7083a5b39e454938fe"} Jul 14 13:28:25.165148 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.164058 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdd87567b-dnjlz"] Jul 14 13:28:25.166661 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.166635 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm"] Jul 14 13:28:25.166787 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.166766 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdd87567b-dnjlz" Jul 14 13:28:25.171110 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.170551 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" Jul 14 13:28:25.171110 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.170592 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 14 13:28:25.171110 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.170659 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-497bq\"" Jul 14 13:28:25.171110 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.171007 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 14 13:28:25.171386 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.171133 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 14 13:28:25.171443 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.171391 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 14 13:28:25.174320 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.174062 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 14 13:28:25.174320 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.174067 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 14 13:28:25.174320 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.174179 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 14 13:28:25.174320 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.174068 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 14 13:28:25.180308 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.180222 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdd87567b-dnjlz"] Jul 14 13:28:25.183794 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.183773 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm"] Jul 14 13:28:25.225467 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.225434 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5f578095-cb4e-4e08-a38b-d2aae3d0373a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6c596dc7db-42mzm\" (UID: \"5f578095-cb4e-4e08-a38b-d2aae3d0373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" Jul 14 13:28:25.225620 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.225481 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f578095-cb4e-4e08-a38b-d2aae3d0373a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6c596dc7db-42mzm\" (UID: \"5f578095-cb4e-4e08-a38b-d2aae3d0373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" Jul 14 13:28:25.225620 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.225531 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5f578095-cb4e-4e08-a38b-d2aae3d0373a-ca\") pod \"cluster-proxy-proxy-agent-6c596dc7db-42mzm\" (UID: \"5f578095-cb4e-4e08-a38b-d2aae3d0373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" Jul 14 13:28:25.225702 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.225616 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/112f8a1c-6691-4b96-8c68-19b56aac441f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-fdd87567b-dnjlz\" (UID: \"112f8a1c-6691-4b96-8c68-19b56aac441f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdd87567b-dnjlz" Jul 14 13:28:25.225702 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.225650 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5f578095-cb4e-4e08-a38b-d2aae3d0373a-hub\") pod \"cluster-proxy-proxy-agent-6c596dc7db-42mzm\" (UID: \"5f578095-cb4e-4e08-a38b-d2aae3d0373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" Jul 14 13:28:25.225702 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.225682 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s522v\" (UniqueName: \"kubernetes.io/projected/112f8a1c-6691-4b96-8c68-19b56aac441f-kube-api-access-s522v\") pod \"managed-serviceaccount-addon-agent-fdd87567b-dnjlz\" (UID: \"112f8a1c-6691-4b96-8c68-19b56aac441f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdd87567b-dnjlz" Jul 14 13:28:25.225794 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.225731 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5f578095-cb4e-4e08-a38b-d2aae3d0373a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6c596dc7db-42mzm\" (UID: \"5f578095-cb4e-4e08-a38b-d2aae3d0373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" Jul 14 13:28:25.225794 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.225756 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtsxf\" (UniqueName: \"kubernetes.io/projected/5f578095-cb4e-4e08-a38b-d2aae3d0373a-kube-api-access-xtsxf\") pod \"cluster-proxy-proxy-agent-6c596dc7db-42mzm\" (UID: \"5f578095-cb4e-4e08-a38b-d2aae3d0373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" Jul 14 13:28:25.327120 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.327065 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/112f8a1c-6691-4b96-8c68-19b56aac441f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-fdd87567b-dnjlz\" (UID: \"112f8a1c-6691-4b96-8c68-19b56aac441f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdd87567b-dnjlz" Jul 14 13:28:25.327120 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.327125 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5f578095-cb4e-4e08-a38b-d2aae3d0373a-hub\") pod \"cluster-proxy-proxy-agent-6c596dc7db-42mzm\" (UID: \"5f578095-cb4e-4e08-a38b-d2aae3d0373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" Jul 14 13:28:25.327354 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.327147 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s522v\" (UniqueName: \"kubernetes.io/projected/112f8a1c-6691-4b96-8c68-19b56aac441f-kube-api-access-s522v\") pod \"managed-serviceaccount-addon-agent-fdd87567b-dnjlz\" (UID: \"112f8a1c-6691-4b96-8c68-19b56aac441f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdd87567b-dnjlz" Jul 14 13:28:25.327354 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.327178 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5f578095-cb4e-4e08-a38b-d2aae3d0373a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6c596dc7db-42mzm\" (UID: \"5f578095-cb4e-4e08-a38b-d2aae3d0373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" Jul 14 13:28:25.327354 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.327237 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtsxf\" (UniqueName: \"kubernetes.io/projected/5f578095-cb4e-4e08-a38b-d2aae3d0373a-kube-api-access-xtsxf\") pod \"cluster-proxy-proxy-agent-6c596dc7db-42mzm\" (UID: \"5f578095-cb4e-4e08-a38b-d2aae3d0373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" Jul 14 13:28:25.327354 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.327307 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5f578095-cb4e-4e08-a38b-d2aae3d0373a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6c596dc7db-42mzm\" (UID: \"5f578095-cb4e-4e08-a38b-d2aae3d0373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" Jul 14 13:28:25.327354 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.327336 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f578095-cb4e-4e08-a38b-d2aae3d0373a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6c596dc7db-42mzm\" (UID: \"5f578095-cb4e-4e08-a38b-d2aae3d0373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" Jul 14 13:28:25.327620 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.327379 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5f578095-cb4e-4e08-a38b-d2aae3d0373a-ca\") pod \"cluster-proxy-proxy-agent-6c596dc7db-42mzm\" (UID: \"5f578095-cb4e-4e08-a38b-d2aae3d0373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" Jul 14 13:28:25.329367 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.329294 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f578095-cb4e-4e08-a38b-d2aae3d0373a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6c596dc7db-42mzm\" (UID: \"5f578095-cb4e-4e08-a38b-d2aae3d0373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" Jul 14 13:28:25.330389 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.330336 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5f578095-cb4e-4e08-a38b-d2aae3d0373a-hub\") pod \"cluster-proxy-proxy-agent-6c596dc7db-42mzm\" (UID: \"5f578095-cb4e-4e08-a38b-d2aae3d0373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" Jul 14 13:28:25.330577 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.330522 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5f578095-cb4e-4e08-a38b-d2aae3d0373a-ca\") pod \"cluster-proxy-proxy-agent-6c596dc7db-42mzm\" (UID: \"5f578095-cb4e-4e08-a38b-d2aae3d0373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" Jul 14 13:28:25.330755 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.330724 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5f578095-cb4e-4e08-a38b-d2aae3d0373a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6c596dc7db-42mzm\" (UID: \"5f578095-cb4e-4e08-a38b-d2aae3d0373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" Jul 14 13:28:25.330849 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.330789 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/112f8a1c-6691-4b96-8c68-19b56aac441f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-fdd87567b-dnjlz\" (UID: \"112f8a1c-6691-4b96-8c68-19b56aac441f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdd87567b-dnjlz" Jul 14 13:28:25.330929 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.330899 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:25.332064 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.332039 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5f578095-cb4e-4e08-a38b-d2aae3d0373a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6c596dc7db-42mzm\" (UID: \"5f578095-cb4e-4e08-a38b-d2aae3d0373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" Jul 14 13:28:25.338038 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.337635 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s522v\" (UniqueName: \"kubernetes.io/projected/112f8a1c-6691-4b96-8c68-19b56aac441f-kube-api-access-s522v\") pod \"managed-serviceaccount-addon-agent-fdd87567b-dnjlz\" (UID: \"112f8a1c-6691-4b96-8c68-19b56aac441f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdd87567b-dnjlz" Jul 14 13:28:25.338038 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.337999 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtsxf\" (UniqueName: \"kubernetes.io/projected/5f578095-cb4e-4e08-a38b-d2aae3d0373a-kube-api-access-xtsxf\") pod \"cluster-proxy-proxy-agent-6c596dc7db-42mzm\" (UID: \"5f578095-cb4e-4e08-a38b-d2aae3d0373a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" Jul 14 13:28:25.433610 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.433582 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:25.495216 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.494843 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdd87567b-dnjlz" Jul 14 13:28:25.502647 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.502574 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" Jul 14 13:28:25.529908 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.529823 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:25.630119 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.630058 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-metrics-tls\") pod \"dns-default-2zzrk\" (UID: \"82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0\") " pod="openshift-dns/dns-default-2zzrk" Jul 14 13:28:25.634016 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.633951 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0-metrics-tls\") pod \"dns-default-2zzrk\" (UID: \"82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0\") " pod="openshift-dns/dns-default-2zzrk" Jul 14 13:28:25.635095 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.635003 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:25.657283 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.657218 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdd87567b-dnjlz"] Jul 14 13:28:25.661371 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:28:25.661333 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod112f8a1c_6691_4b96_8c68_19b56aac441f.slice/crio-826b12272bc7cf2bbbf86c5133257bb0fa59b73687cbd1b80fd650f395621beb WatchSource:0}: Error finding container 826b12272bc7cf2bbbf86c5133257bb0fa59b73687cbd1b80fd650f395621beb: Status 404 returned error can't find the container with id 826b12272bc7cf2bbbf86c5133257bb0fa59b73687cbd1b80fd650f395621beb Jul 14 13:28:25.681690 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.681581 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm"] Jul 14 13:28:25.685213 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:28:25.685176 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f578095_cb4e_4e08_a38b_d2aae3d0373a.slice/crio-a63c3f05828444acc3616553f67041eb86deb605ba47dc5e99cbeef54c940e56 WatchSource:0}: Error finding container a63c3f05828444acc3616553f67041eb86deb605ba47dc5e99cbeef54c940e56: Status 404 returned error can't find the container with id a63c3f05828444acc3616553f67041eb86deb605ba47dc5e99cbeef54c940e56 Jul 14 13:28:25.823674 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.823634 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xrgrj" event={"ID":"50a391d6-3c2b-478d-b005-4bbd65bdec74","Type":"ContainerStarted","Data":"b3d1750cde0187f00e171380638d17794dbb590ec5a5e36a46b71f5bd16a4771"} Jul 14 13:28:25.823840 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.823709 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xrgrj" event={"ID":"50a391d6-3c2b-478d-b005-4bbd65bdec74","Type":"ContainerStarted","Data":"a0f582f02b3e05ec00e968ab5ad46b64b8dee9b3263c9fe69f7f5bdd16683e5d"} Jul 14 13:28:25.825221 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.825193 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdd87567b-dnjlz" event={"ID":"112f8a1c-6691-4b96-8c68-19b56aac441f","Type":"ContainerStarted","Data":"826b12272bc7cf2bbbf86c5133257bb0fa59b73687cbd1b80fd650f395621beb"} Jul 14 13:28:25.826828 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.826793 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" event={"ID":"5f578095-cb4e-4e08-a38b-d2aae3d0373a","Type":"ContainerStarted","Data":"a63c3f05828444acc3616553f67041eb86deb605ba47dc5e99cbeef54c940e56"} Jul 14 13:28:25.842160 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.842104 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xrgrj" podStartSLOduration=65.861924245 podStartE2EDuration="1m6.842061141s" totalImagesPullingTime="980.136896ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:27:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:25.841709485 +0000 UTC m=+69.368860122" watchObservedRunningTime="2026-07-14 13:28:25.842061141 +0000 UTC m=+69.369211778" Jul 14 13:28:25.918725 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.918687 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6rczr\"" Jul 14 13:28:25.926098 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:25.926051 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2zzrk" Jul 14 13:28:26.077237 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:26.077203 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2zzrk"] Jul 14 13:28:26.080706 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:28:26.080679 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82d9fb6c_5cea_4c1f_8e02_8fe701a4d0a0.slice/crio-bf863ca35a693b4bd880f917feeb13e016f5eda5144faf1ef15fd54869bf8da7 WatchSource:0}: Error finding container bf863ca35a693b4bd880f917feeb13e016f5eda5144faf1ef15fd54869bf8da7: Status 404 returned error can't find the container with id bf863ca35a693b4bd880f917feeb13e016f5eda5144faf1ef15fd54869bf8da7 Jul 14 13:28:26.480133 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:26.480055 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:26.565142 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:26.565009 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:26.678301 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:26.678265 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:26.837476 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:26.836601 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2zzrk" event={"ID":"82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0","Type":"ContainerStarted","Data":"bf863ca35a693b4bd880f917feeb13e016f5eda5144faf1ef15fd54869bf8da7"} Jul 14 13:28:26.885620 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:26.885585 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:26.985444 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:26.985407 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:27.827524 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:27.827488 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:28.623194 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:28.623152 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:28.724535 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:28.724494 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:28.791692 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:28.791635 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:29.209463 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:29.209423 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:29.283858 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:29.283819 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:29.415586 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:29.415550 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:31.729354 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:31.729304 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:31.832725 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:31.832693 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:32.036260 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:32.036214 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:32.502645 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:32.502615 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:32.504935 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:32.504902 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f321dd9-1f3d-42c0-b6eb-05130c73d8b9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jmjw8\" (UID: \"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9\") " pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:32.682986 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:32.682952 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tbwpd\"" Jul 14 13:28:32.691374 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:32.691338 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jmjw8" Jul 14 13:28:33.253691 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:33.253565 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jmjw8"] Jul 14 13:28:33.257615 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:28:33.257584 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f321dd9_1f3d_42c0_b6eb_05130c73d8b9.slice/crio-196fbb5870d234cf20c7c10be860641d7bfc85280fdf3bc1035b60898d31d347 WatchSource:0}: Error finding container 196fbb5870d234cf20c7c10be860641d7bfc85280fdf3bc1035b60898d31d347: Status 404 returned error can't find the container with id 196fbb5870d234cf20c7c10be860641d7bfc85280fdf3bc1035b60898d31d347 Jul 14 13:28:33.873164 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:33.873050 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jmjw8" event={"ID":"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9","Type":"ContainerStarted","Data":"196fbb5870d234cf20c7c10be860641d7bfc85280fdf3bc1035b60898d31d347"} Jul 14 13:28:33.875925 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:33.875497 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdd87567b-dnjlz" event={"ID":"112f8a1c-6691-4b96-8c68-19b56aac441f","Type":"ContainerStarted","Data":"bd0455eab451bb85eb8e2b4511546e5215e0f23677d5ec15d0d6900bd4db6593"} Jul 14 13:28:33.877697 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:33.877668 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wnnf2" event={"ID":"1bf7c74f-a569-44e9-98cb-ee2255c6067e","Type":"ContainerStarted","Data":"21249e2cfd8681ee3ac61db097f3f06922aa757e4f1d3d926774e77bb10e864b"} Jul 14 13:28:33.878164 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:33.878144 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:28:33.880764 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:33.880738 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z9kpt" event={"ID":"92645e2c-0c99-4eca-930a-686d53f52229","Type":"ContainerStarted","Data":"1a8d05e5c3d580246f610babc1b97602424ab2f09490599175e58492e2edf8a3"} Jul 14 13:28:33.884810 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:33.884785 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" event={"ID":"5f578095-cb4e-4e08-a38b-d2aae3d0373a","Type":"ContainerStarted","Data":"f40ab0f9ca82f705c561c95b91a6f16a398f5bc7ed0995eb86822b06001a84fc"} Jul 14 13:28:33.884921 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:33.884818 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" event={"ID":"5f578095-cb4e-4e08-a38b-d2aae3d0373a","Type":"ContainerStarted","Data":"061c8a2729ae31d88f2dc0d0ad7244048674b1f0272a9a0a49c8e33db911aa5e"} Jul 14 13:28:33.884921 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:33.884836 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" event={"ID":"5f578095-cb4e-4e08-a38b-d2aae3d0373a","Type":"ContainerStarted","Data":"81eb9486eb54df470801e417e120b9e89f2fd8aaaeda38e4a7bc9adf8492e1ba"} Jul 14 13:28:33.890741 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:33.890716 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2zzrk" event={"ID":"82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0","Type":"ContainerStarted","Data":"16e5e4cf4d55bc72b21e2f7f103c764706560d4dc74bb21401a8356513f24348"} Jul 14 13:28:33.890853 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:33.890767 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2zzrk" event={"ID":"82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0","Type":"ContainerStarted","Data":"094d2d8605b6fd771e0f520629bb31ab894a47b516b06ec635a95112943e711e"} Jul 14 13:28:33.891301 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:33.891281 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2zzrk" Jul 14 13:28:33.893838 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:33.893793 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fdd87567b-dnjlz" podStartSLOduration=1.469788447 podStartE2EDuration="8.893776402s" totalImagesPullingTime="7.423987955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:28:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:33.89359646 +0000 UTC m=+77.420747096" watchObservedRunningTime="2026-07-14 13:28:33.893776402 +0000 UTC m=+77.420927038" Jul 14 13:28:33.914953 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:33.914239 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2zzrk" podStartSLOduration=33.907859035 podStartE2EDuration="40.914216897s" totalImagesPullingTime="7.006357862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:33.912959413 +0000 UTC m=+77.440110062" watchObservedRunningTime="2026-07-14 13:28:33.914216897 +0000 UTC m=+77.441367535" Jul 14 13:28:33.944925 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:33.943924 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-z9kpt" podStartSLOduration=66.171535745 podStartE2EDuration="1m14.943902704s" totalImagesPullingTime="8.772366959s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:33.943459063 +0000 UTC m=+77.470609700" watchObservedRunningTime="2026-07-14 13:28:33.943902704 +0000 UTC m=+77.471053342" Jul 14 13:28:33.996567 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:33.995991 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c596dc7db-42mzm" podStartSLOduration=1.546715174 podStartE2EDuration="8.995970661s" totalImagesPullingTime="7.449255487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:28:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:33.994422133 +0000 UTC m=+77.521572770" watchObservedRunningTime="2026-07-14 13:28:33.995970661 +0000 UTC m=+77.523121298" Jul 14 13:28:34.806243 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:34.806200 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:36.058922 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:36.058878 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:36.119989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:36.119953 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:37.253813 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:37.253757 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:37.376699 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:37.376666 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:37.479659 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:37.479628 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:37.906668 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:37.906578 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jmjw8" event={"ID":"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9","Type":"ContainerStarted","Data":"2bd246f1e35cc73325d60eab924797513a0dcf0feebea416b55b46e17f6b0fc6"} Jul 14 13:28:37.906811 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:37.906703 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jmjw8" event={"ID":"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9","Type":"ContainerStarted","Data":"c78b5642e6b783e001930ce4d1ecd81fa54fd0f554043e22fc99961450cab7a2"} Jul 14 13:28:38.878428 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:38.878278 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:39.005257 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.005216 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:39.113524 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.113436 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:39.157284 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.157014 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wnnf2" podStartSLOduration=71.721043186 podStartE2EDuration="1m20.15699129s" totalImagesPullingTime="8.435948104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:27:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:34.012625292 +0000 UTC m=+77.539775928" watchObservedRunningTime="2026-07-14 13:28:39.15699129 +0000 UTC m=+82.684141927" Jul 14 13:28:39.157284 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.157193 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rtnvv"] Jul 14 13:28:39.162153 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.162127 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.165427 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.165167 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zg27v\"" Jul 14 13:28:39.165427 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.165198 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 14 13:28:39.166441 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.166410 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 14 13:28:39.166441 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.166434 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 14 13:28:39.166732 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.166712 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 14 13:28:39.166926 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.166814 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 14 13:28:39.166926 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.166829 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 14 13:28:39.194669 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.194630 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:39.255877 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.255826 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1f971f01-724e-47f5-af52-16610accaf3a-root\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.256061 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.255899 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f971f01-724e-47f5-af52-16610accaf3a-metrics-client-ca\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.256061 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.255932 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1f971f01-724e-47f5-af52-16610accaf3a-node-exporter-tls\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.256061 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.255968 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f971f01-724e-47f5-af52-16610accaf3a-sys\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.256061 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.256007 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z95nb\" (UniqueName: \"kubernetes.io/projected/1f971f01-724e-47f5-af52-16610accaf3a-kube-api-access-z95nb\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.256061 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.256036 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1f971f01-724e-47f5-af52-16610accaf3a-node-exporter-accelerators-collector-config\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.256354 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.256065 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1f971f01-724e-47f5-af52-16610accaf3a-node-exporter-wtmp\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.256354 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.256156 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f971f01-724e-47f5-af52-16610accaf3a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.256354 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.256187 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1f971f01-724e-47f5-af52-16610accaf3a-node-exporter-textfile\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.258462 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.258430 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:39.357296 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.357253 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1f971f01-724e-47f5-af52-16610accaf3a-root\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.357522 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.357337 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f971f01-724e-47f5-af52-16610accaf3a-metrics-client-ca\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.357522 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.357369 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1f971f01-724e-47f5-af52-16610accaf3a-node-exporter-tls\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.357522 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.357378 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1f971f01-724e-47f5-af52-16610accaf3a-root\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.357522 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.357414 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f971f01-724e-47f5-af52-16610accaf3a-sys\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.357522 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.357460 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z95nb\" (UniqueName: \"kubernetes.io/projected/1f971f01-724e-47f5-af52-16610accaf3a-kube-api-access-z95nb\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.357522 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.357495 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1f971f01-724e-47f5-af52-16610accaf3a-node-exporter-accelerators-collector-config\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.357832 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.357526 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1f971f01-724e-47f5-af52-16610accaf3a-node-exporter-wtmp\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.357832 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.357570 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f971f01-724e-47f5-af52-16610accaf3a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.357832 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.357598 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1f971f01-724e-47f5-af52-16610accaf3a-node-exporter-textfile\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.357979 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.357887 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1f971f01-724e-47f5-af52-16610accaf3a-node-exporter-textfile\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.357979 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.357951 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1f971f01-724e-47f5-af52-16610accaf3a-node-exporter-wtmp\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.358097 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.358008 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f971f01-724e-47f5-af52-16610accaf3a-sys\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.358097 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.358032 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f971f01-724e-47f5-af52-16610accaf3a-metrics-client-ca\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.358325 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.358303 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1f971f01-724e-47f5-af52-16610accaf3a-node-exporter-accelerators-collector-config\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.360641 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.360617 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f971f01-724e-47f5-af52-16610accaf3a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.360909 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.360859 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1f971f01-724e-47f5-af52-16610accaf3a-node-exporter-tls\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.370510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.370484 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z95nb\" (UniqueName: \"kubernetes.io/projected/1f971f01-724e-47f5-af52-16610accaf3a-kube-api-access-z95nb\") pod \"node-exporter-rtnvv\" (UID: \"1f971f01-724e-47f5-af52-16610accaf3a\") " pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.383541 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.383501 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:39.475391 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.475304 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rtnvv" Jul 14 13:28:39.688665 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.688617 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:39.787614 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.787577 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:39.934871 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:28:39.934841 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f971f01_724e_47f5_af52_16610accaf3a.slice/crio-3f5199c40b40145bd0fcd5b8c7dca1fdb07f3fa3dc5eb7e09930a55825981acc WatchSource:0}: Error finding container 3f5199c40b40145bd0fcd5b8c7dca1fdb07f3fa3dc5eb7e09930a55825981acc: Status 404 returned error can't find the container with id 3f5199c40b40145bd0fcd5b8c7dca1fdb07f3fa3dc5eb7e09930a55825981acc Jul 14 13:28:39.972804 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:39.972775 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:40.120170 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:40.120062 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:40.220165 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:40.220127 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:40.341421 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:40.341386 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:40.440139 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:40.440029 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:40.919292 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:40.919256 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rtnvv" event={"ID":"1f971f01-724e-47f5-af52-16610accaf3a","Type":"ContainerStarted","Data":"3f5199c40b40145bd0fcd5b8c7dca1fdb07f3fa3dc5eb7e09930a55825981acc"} Jul 14 13:28:40.921299 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:40.921273 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jmjw8" event={"ID":"6f321dd9-1f3d-42c0-b6eb-05130c73d8b9","Type":"ContainerStarted","Data":"c8aeaa234da6f0d94c613e01f1cc0948cd48622b1fdb71dd2da29d07cc85039b"} Jul 14 13:28:40.940809 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:40.940760 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jmjw8" podStartSLOduration=34.334934726 podStartE2EDuration="40.940746052s" totalImagesPullingTime="6.605811326s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:28:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:40.940594811 +0000 UTC m=+84.467745447" watchObservedRunningTime="2026-07-14 13:28:40.940746052 +0000 UTC m=+84.467896687" Jul 14 13:28:41.272959 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:41.272914 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:41.926152 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:41.926118 2583 generic.go:356] "Generic (PLEG): container finished" podID="1f971f01-724e-47f5-af52-16610accaf3a" containerID="fa59f95d56ba258aa5557a7ec90fdff2ea3504b8ac00107fc70ae7d4587edbdf" exitCode=0 Jul 14 13:28:41.926350 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:41.926211 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rtnvv" event={"ID":"1f971f01-724e-47f5-af52-16610accaf3a","Type":"ContainerDied","Data":"fa59f95d56ba258aa5557a7ec90fdff2ea3504b8ac00107fc70ae7d4587edbdf"} Jul 14 13:28:42.241996 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:42.241869 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:42.306556 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:42.306519 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:42.437552 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:42.437519 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:42.535817 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:42.535785 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:42.649867 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:42.649796 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:42.710131 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:42.710072 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:42.931997 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:42.931910 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rtnvv" event={"ID":"1f971f01-724e-47f5-af52-16610accaf3a","Type":"ContainerStarted","Data":"734c5c2ebdfa20fa3e44e11f85c141fe966f819b3dd9c330e85c292b29517e1a"} Jul 14 13:28:42.931997 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:42.931961 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rtnvv" event={"ID":"1f971f01-724e-47f5-af52-16610accaf3a","Type":"ContainerStarted","Data":"afd9cc1c5383c828e1289795fa9a2de764b2ac4ae3f2738aa3467b49fd421d7d"} Jul 14 13:28:42.957867 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:42.957820 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rtnvv" podStartSLOduration=2.808613061 podStartE2EDuration="3.957804245s" totalImagesPullingTime="1.087599393s" totalInitContainerRuntime="61.591791ms" isStatefulPod=true podCreationTimestamp="2026-07-14 13:28:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:42.957183492 +0000 UTC m=+86.484334127" watchObservedRunningTime="2026-07-14 13:28:42.957804245 +0000 UTC m=+86.484954880" Jul 14 13:28:43.778104 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:43.778046 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:43.880461 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:43.880431 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:43.960892 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:43.960858 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:44.197199 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.197110 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:44.358930 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.358881 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn"] Jul 14 13:28:44.362126 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.362102 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.365000 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.364977 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 14 13:28:44.365149 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.365108 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 14 13:28:44.365149 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.365128 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 14 13:28:44.365264 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.365107 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 14 13:28:44.365414 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.365398 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-pwbw5\"" Jul 14 13:28:44.365474 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.365460 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 14 13:28:44.371576 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.371555 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 14 13:28:44.382117 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.380144 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn"] Jul 14 13:28:44.496055 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.496026 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f52836e6-99fb-4f20-b546-df96573d560f-telemeter-client-tls\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.496240 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.496059 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f52836e6-99fb-4f20-b546-df96573d560f-serving-certs-ca-bundle\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.496240 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.496105 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgs7v\" (UniqueName: \"kubernetes.io/projected/f52836e6-99fb-4f20-b546-df96573d560f-kube-api-access-mgs7v\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.496240 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.496149 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f52836e6-99fb-4f20-b546-df96573d560f-secret-telemeter-client\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.496240 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.496225 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f52836e6-99fb-4f20-b546-df96573d560f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.496388 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.496256 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f52836e6-99fb-4f20-b546-df96573d560f-metrics-client-ca\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.496388 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.496276 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f52836e6-99fb-4f20-b546-df96573d560f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.496388 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.496294 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f52836e6-99fb-4f20-b546-df96573d560f-federate-client-tls\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.597237 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.597197 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgs7v\" (UniqueName: \"kubernetes.io/projected/f52836e6-99fb-4f20-b546-df96573d560f-kube-api-access-mgs7v\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.597237 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.597235 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f52836e6-99fb-4f20-b546-df96573d560f-secret-telemeter-client\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.597501 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.597271 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f52836e6-99fb-4f20-b546-df96573d560f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.597501 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.597312 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f52836e6-99fb-4f20-b546-df96573d560f-metrics-client-ca\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.597501 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.597342 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f52836e6-99fb-4f20-b546-df96573d560f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.597501 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.597370 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f52836e6-99fb-4f20-b546-df96573d560f-federate-client-tls\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.597696 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.597413 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f52836e6-99fb-4f20-b546-df96573d560f-telemeter-client-tls\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.597696 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.597657 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f52836e6-99fb-4f20-b546-df96573d560f-serving-certs-ca-bundle\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.598557 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.598524 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f52836e6-99fb-4f20-b546-df96573d560f-serving-certs-ca-bundle\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.598661 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.598524 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f52836e6-99fb-4f20-b546-df96573d560f-metrics-client-ca\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.598661 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.598605 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f52836e6-99fb-4f20-b546-df96573d560f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.600132 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.600107 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f52836e6-99fb-4f20-b546-df96573d560f-telemeter-client-tls\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.600210 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.600195 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f52836e6-99fb-4f20-b546-df96573d560f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.600400 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.600379 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f52836e6-99fb-4f20-b546-df96573d560f-secret-telemeter-client\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.600436 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.600397 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f52836e6-99fb-4f20-b546-df96573d560f-federate-client-tls\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.607617 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.607590 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgs7v\" (UniqueName: \"kubernetes.io/projected/f52836e6-99fb-4f20-b546-df96573d560f-kube-api-access-mgs7v\") pod \"telemeter-client-78fbd8f94c-lpzzn\" (UID: \"f52836e6-99fb-4f20-b546-df96573d560f\") " pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.671328 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.671282 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" Jul 14 13:28:44.817449 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.817211 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn"] Jul 14 13:28:44.819571 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:28:44.819542 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf52836e6_99fb_4f20_b546_df96573d560f.slice/crio-d165537a2f8dc9eed3299d08c675b33be78fd92432196954d7c122289cf1a1d6 WatchSource:0}: Error finding container d165537a2f8dc9eed3299d08c675b33be78fd92432196954d7c122289cf1a1d6: Status 404 returned error can't find the container with id d165537a2f8dc9eed3299d08c675b33be78fd92432196954d7c122289cf1a1d6 Jul 14 13:28:44.900037 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.900010 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2zzrk" Jul 14 13:28:44.940958 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:44.940907 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" event={"ID":"f52836e6-99fb-4f20-b546-df96573d560f","Type":"ContainerStarted","Data":"d165537a2f8dc9eed3299d08c675b33be78fd92432196954d7c122289cf1a1d6"} Jul 14 13:28:45.421986 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.421951 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:45.422186 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.421998 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 13:28:45.425362 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.425342 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.428770 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.428748 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 14 13:28:45.429258 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.429230 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5agnm3a4tti59\"" Jul 14 13:28:45.429355 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.429236 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 14 13:28:45.429607 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.429582 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 14 13:28:45.429607 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.429590 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 14 13:28:45.429757 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.429647 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 14 13:28:45.429757 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.429654 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 14 13:28:45.429757 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.429651 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 14 13:28:45.429757 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.429593 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 14 13:28:45.429757 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.429728 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 14 13:28:45.430040 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.430020 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 14 13:28:45.430224 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.430209 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bhlr2\"" Jul 14 13:28:45.430298 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.430269 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 14 13:28:45.431053 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.431023 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 14 13:28:45.431152 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.431073 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 14 13:28:45.431230 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.431100 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 14 13:28:45.432532 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.432514 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 14 13:28:45.436964 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.436945 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 13:28:45.521450 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.521420 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.521450 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.521450 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.521666 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.521468 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.521666 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.521490 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-config-out\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.521666 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.521515 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.521666 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.521541 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-config\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.521666 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.521571 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.521666 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.521640 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.521666 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.521660 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.521993 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.521681 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.521993 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.521702 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.521993 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.521744 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.521993 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.521792 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.521993 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.521849 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.521993 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.521875 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.521993 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.521899 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcqhr\" (UniqueName: \"kubernetes.io/projected/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-kube-api-access-bcqhr\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.521993 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.521927 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.521993 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.521953 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.521993 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.521978 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.522346 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.522021 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-web-config\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.623389 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.623356 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-config-out\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.623389 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.623389 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.623617 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.623412 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-config\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.623617 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.623447 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.623617 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.623478 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.623617 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.623505 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.623617 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.623528 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.623905 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.623666 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.623905 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.623701 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.623905 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.623730 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.623905 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.623793 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.623905 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.623824 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.623905 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.623847 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcqhr\" (UniqueName: \"kubernetes.io/projected/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-kube-api-access-bcqhr\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.623905 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.623876 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.623905 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.623904 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.624334 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.623928 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.624334 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.623952 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-web-config\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.624334 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.623982 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.624334 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.624005 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.624334 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.624028 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.624334 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.624329 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.624982 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.624652 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.624982 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.624743 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.624982 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.624856 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.625226 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.625004 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.626036 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.626015 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.626932 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.626904 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.627348 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.627321 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-config-out\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.627429 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.627365 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-config\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.627654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.627614 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.627852 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.627828 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.627946 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.627930 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-web-config\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.628543 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.628495 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.628644 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.628536 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.628705 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.628656 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.629569 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.629546 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.629669 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.629627 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.629742 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.629726 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.630196 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.630176 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.633556 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.633538 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcqhr\" (UniqueName: \"kubernetes.io/projected/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-kube-api-access-bcqhr\") pod \"prometheus-k8s-0\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.662238 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.662213 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:45.724610 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.724535 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:45.735515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.735492 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:45.899241 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.899210 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 13:28:45.902437 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:28:45.902407 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ecb9c2a_202a_4cf8_be0d_82eb828138b5.slice/crio-a8d4a8625d687ad960e91fc45bde309c63e235ac98fa5b92479af7894366d3bd WatchSource:0}: Error finding container a8d4a8625d687ad960e91fc45bde309c63e235ac98fa5b92479af7894366d3bd: Status 404 returned error can't find the container with id a8d4a8625d687ad960e91fc45bde309c63e235ac98fa5b92479af7894366d3bd Jul 14 13:28:45.950072 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:45.950039 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ecb9c2a-202a-4cf8-be0d-82eb828138b5","Type":"ContainerStarted","Data":"a8d4a8625d687ad960e91fc45bde309c63e235ac98fa5b92479af7894366d3bd"} Jul 14 13:28:47.641326 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:47.641282 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:47.706631 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:47.706589 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:47.960397 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:47.960292 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" event={"ID":"f52836e6-99fb-4f20-b546-df96573d560f","Type":"ContainerStarted","Data":"530dc7d581ce89257bd16e312828d95e1d42d29527db6dfbb7cd9c2c0e0ad0e5"} Jul 14 13:28:47.960397 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:47.960356 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" event={"ID":"f52836e6-99fb-4f20-b546-df96573d560f","Type":"ContainerStarted","Data":"0ffab64457ae9e5c357fd60631f3ae8b97a975d99db9ab098eaecb2a07f947c6"} Jul 14 13:28:47.960397 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:47.960387 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" event={"ID":"f52836e6-99fb-4f20-b546-df96573d560f","Type":"ContainerStarted","Data":"d3118883bb125002ce46f2f5b4b8d1104008b0339b55e44221c839246a929eef"} Jul 14 13:28:47.961793 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:47.961772 2583 generic.go:356] "Generic (PLEG): container finished" podID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerID="9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f" exitCode=0 Jul 14 13:28:47.961889 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:47.961826 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ecb9c2a-202a-4cf8-be0d-82eb828138b5","Type":"ContainerDied","Data":"9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f"} Jul 14 13:28:47.990021 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:47.989968 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-78fbd8f94c-lpzzn" podStartSLOduration=1.518945803 podStartE2EDuration="3.989952203s" totalImagesPullingTime="2.4710064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:28:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:47.988032181 +0000 UTC m=+91.515182816" watchObservedRunningTime="2026-07-14 13:28:47.989952203 +0000 UTC m=+91.517102839" Jul 14 13:28:49.554147 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:49.554109 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:49.622204 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:49.622162 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:50.676282 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:50.676244 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:50.752584 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:50.752542 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:50.975746 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:50.975704 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ecb9c2a-202a-4cf8-be0d-82eb828138b5","Type":"ContainerStarted","Data":"80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4"} Jul 14 13:28:51.981413 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:51.981381 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ecb9c2a-202a-4cf8-be0d-82eb828138b5","Type":"ContainerStarted","Data":"314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698"} Jul 14 13:28:52.180833 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:52.180797 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:52.253071 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:52.252999 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:52.988438 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:52.988398 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ecb9c2a-202a-4cf8-be0d-82eb828138b5","Type":"ContainerStarted","Data":"bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13"} Jul 14 13:28:52.988778 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:52.988497 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ecb9c2a-202a-4cf8-be0d-82eb828138b5","Type":"ContainerStarted","Data":"5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1"} Jul 14 13:28:52.988778 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:52.988525 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ecb9c2a-202a-4cf8-be0d-82eb828138b5","Type":"ContainerStarted","Data":"5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d"} Jul 14 13:28:52.988778 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:52.988550 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ecb9c2a-202a-4cf8-be0d-82eb828138b5","Type":"ContainerStarted","Data":"e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8"} Jul 14 13:28:53.032854 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:53.032786 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.017852981 podStartE2EDuration="8.03276259s" totalImagesPullingTime="5.992539719s" totalInitContainerRuntime="22.36989ms" isStatefulPod=false podCreationTimestamp="2026-07-14 13:28:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:53.030420985 +0000 UTC m=+96.557571622" watchObservedRunningTime="2026-07-14 13:28:53.03276259 +0000 UTC m=+96.559913226" Jul 14 13:28:54.686653 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:54.686619 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:54.794937 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:54.794900 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:55.736469 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:55.736432 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:28:58.634971 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:58.634883 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:58.735203 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:58.735168 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:59.206862 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:59.206826 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:59.274538 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:59.274494 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:28:59.920794 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:28:59.920759 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:29:05.901159 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:29:05.901129 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wnnf2" Jul 14 13:29:14.214970 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:29:14.214931 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:29:14.304100 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:29:14.304035 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:29:45.735897 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:29:45.735842 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:29:45.755381 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:29:45.755358 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:29:46.205763 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:29:46.205737 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:29:47.416335 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:29:47.416300 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:29:55.281618 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:29:55.281581 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:29:55.382494 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:29:55.382451 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:30:03.831832 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:03.831796 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 13:30:03.832374 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:03.832108 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="prometheus" containerID="cri-o://80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4" gracePeriod=600 Jul 14 13:30:03.832374 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:03.832148 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="kube-rbac-proxy-thanos" containerID="cri-o://bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13" gracePeriod=600 Jul 14 13:30:03.832374 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:03.832153 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="thanos-sidecar" containerID="cri-o://e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8" gracePeriod=600 Jul 14 13:30:03.832374 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:03.832176 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="config-reloader" containerID="cri-o://314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698" gracePeriod=600 Jul 14 13:30:03.832374 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:03.832172 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="kube-rbac-proxy-web" containerID="cri-o://5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d" gracePeriod=600 Jul 14 13:30:03.832374 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:03.832114 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="kube-rbac-proxy" containerID="cri-o://5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1" gracePeriod=600 Jul 14 13:30:04.077883 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.077856 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.243126 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243025 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-metrics-client-certs\") pod \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " Jul 14 13:30:04.243126 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243101 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-trusted-ca-bundle\") pod \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " Jul 14 13:30:04.243345 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243135 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-rulefiles-1\") pod \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " Jul 14 13:30:04.243345 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243154 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-web-config\") pod \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " Jul 14 13:30:04.243345 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243175 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-prometheus-k8s-tls\") pod \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " Jul 14 13:30:04.243345 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243199 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-rulefiles-2\") pod \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " Jul 14 13:30:04.243345 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243228 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-configmap-metrics-client-ca\") pod \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " Jul 14 13:30:04.243345 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243254 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-kube-rbac-proxy\") pod \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " Jul 14 13:30:04.243345 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243283 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-config\") pod \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " Jul 14 13:30:04.243345 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243315 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-configmap-serving-certs-ca-bundle\") pod \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " Jul 14 13:30:04.243345 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243343 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " Jul 14 13:30:04.243819 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243371 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-thanos-prometheus-http-client-file\") pod \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " Jul 14 13:30:04.243819 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243399 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-grpc-tls\") pod \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " Jul 14 13:30:04.243819 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243455 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcqhr\" (UniqueName: \"kubernetes.io/projected/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-kube-api-access-bcqhr\") pod \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " Jul 14 13:30:04.243819 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243495 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " Jul 14 13:30:04.243819 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243522 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-configmap-kubelet-serving-ca-bundle\") pod \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " Jul 14 13:30:04.243819 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243534 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "1ecb9c2a-202a-4cf8-be0d-82eb828138b5" (UID: "1ecb9c2a-202a-4cf8-be0d-82eb828138b5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:30:04.243819 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243556 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-config-out\") pod \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " Jul 14 13:30:04.243819 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243593 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-rulefiles-0\") pod \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " Jul 14 13:30:04.243819 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243612 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "1ecb9c2a-202a-4cf8-be0d-82eb828138b5" (UID: "1ecb9c2a-202a-4cf8-be0d-82eb828138b5"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:30:04.243819 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243634 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-db\") pod \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " Jul 14 13:30:04.243819 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243660 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-tls-assets\") pod \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\" (UID: \"1ecb9c2a-202a-4cf8-be0d-82eb828138b5\") " Jul 14 13:30:04.244388 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243887 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:30:04.244388 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243905 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:30:04.244388 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243908 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "1ecb9c2a-202a-4cf8-be0d-82eb828138b5" (UID: "1ecb9c2a-202a-4cf8-be0d-82eb828138b5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:30:04.244388 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.243915 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "1ecb9c2a-202a-4cf8-be0d-82eb828138b5" (UID: "1ecb9c2a-202a-4cf8-be0d-82eb828138b5"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:30:04.244388 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.244337 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "1ecb9c2a-202a-4cf8-be0d-82eb828138b5" (UID: "1ecb9c2a-202a-4cf8-be0d-82eb828138b5"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:30:04.245904 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.245874 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "1ecb9c2a-202a-4cf8-be0d-82eb828138b5" (UID: "1ecb9c2a-202a-4cf8-be0d-82eb828138b5"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:30:04.246309 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.246277 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "1ecb9c2a-202a-4cf8-be0d-82eb828138b5" (UID: "1ecb9c2a-202a-4cf8-be0d-82eb828138b5"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:30:04.246504 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.246471 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-kube-api-access-bcqhr" (OuterVolumeSpecName: "kube-api-access-bcqhr") pod "1ecb9c2a-202a-4cf8-be0d-82eb828138b5" (UID: "1ecb9c2a-202a-4cf8-be0d-82eb828138b5"). InnerVolumeSpecName "kube-api-access-bcqhr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:30:04.246622 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.246567 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1ecb9c2a-202a-4cf8-be0d-82eb828138b5" (UID: "1ecb9c2a-202a-4cf8-be0d-82eb828138b5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:30:04.246680 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.246629 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-config-out" (OuterVolumeSpecName: "config-out") pod "1ecb9c2a-202a-4cf8-be0d-82eb828138b5" (UID: "1ecb9c2a-202a-4cf8-be0d-82eb828138b5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:30:04.246987 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.246963 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "1ecb9c2a-202a-4cf8-be0d-82eb828138b5" (UID: "1ecb9c2a-202a-4cf8-be0d-82eb828138b5"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:30:04.247061 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.246970 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "1ecb9c2a-202a-4cf8-be0d-82eb828138b5" (UID: "1ecb9c2a-202a-4cf8-be0d-82eb828138b5"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:30:04.247267 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.247242 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "1ecb9c2a-202a-4cf8-be0d-82eb828138b5" (UID: "1ecb9c2a-202a-4cf8-be0d-82eb828138b5"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:30:04.247703 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.247674 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "1ecb9c2a-202a-4cf8-be0d-82eb828138b5" (UID: "1ecb9c2a-202a-4cf8-be0d-82eb828138b5"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:30:04.247861 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.247835 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "1ecb9c2a-202a-4cf8-be0d-82eb828138b5" (UID: "1ecb9c2a-202a-4cf8-be0d-82eb828138b5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:30:04.248159 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.248138 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-config" (OuterVolumeSpecName: "config") pod "1ecb9c2a-202a-4cf8-be0d-82eb828138b5" (UID: "1ecb9c2a-202a-4cf8-be0d-82eb828138b5"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:30:04.248586 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.248568 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "1ecb9c2a-202a-4cf8-be0d-82eb828138b5" (UID: "1ecb9c2a-202a-4cf8-be0d-82eb828138b5"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:30:04.248953 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.248926 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "1ecb9c2a-202a-4cf8-be0d-82eb828138b5" (UID: "1ecb9c2a-202a-4cf8-be0d-82eb828138b5"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:30:04.249043 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.249030 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "1ecb9c2a-202a-4cf8-be0d-82eb828138b5" (UID: "1ecb9c2a-202a-4cf8-be0d-82eb828138b5"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:30:04.256778 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.256748 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-web-config" (OuterVolumeSpecName: "web-config") pod "1ecb9c2a-202a-4cf8-be0d-82eb828138b5" (UID: "1ecb9c2a-202a-4cf8-be0d-82eb828138b5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:30:04.258204 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.258179 2583 generic.go:356] "Generic (PLEG): container finished" podID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerID="bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13" exitCode=0 Jul 14 13:30:04.258204 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.258204 2583 generic.go:356] "Generic (PLEG): container finished" podID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerID="5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1" exitCode=0 Jul 14 13:30:04.258348 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.258211 2583 generic.go:356] "Generic (PLEG): container finished" podID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerID="5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d" exitCode=0 Jul 14 13:30:04.258348 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.258218 2583 generic.go:356] "Generic (PLEG): container finished" podID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerID="e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8" exitCode=0 Jul 14 13:30:04.258348 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.258224 2583 generic.go:356] "Generic (PLEG): container finished" podID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerID="314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698" exitCode=0 Jul 14 13:30:04.258348 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.258230 2583 generic.go:356] "Generic (PLEG): container finished" podID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerID="80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4" exitCode=0 Jul 14 13:30:04.258348 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.258254 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ecb9c2a-202a-4cf8-be0d-82eb828138b5","Type":"ContainerDied","Data":"bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13"} Jul 14 13:30:04.258348 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.258269 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.258348 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.258321 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ecb9c2a-202a-4cf8-be0d-82eb828138b5","Type":"ContainerDied","Data":"5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1"} Jul 14 13:30:04.258348 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.258351 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ecb9c2a-202a-4cf8-be0d-82eb828138b5","Type":"ContainerDied","Data":"5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d"} Jul 14 13:30:04.258760 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.258379 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ecb9c2a-202a-4cf8-be0d-82eb828138b5","Type":"ContainerDied","Data":"e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8"} Jul 14 13:30:04.258760 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.258401 2583 scope.go:117] "RemoveContainer" containerID="bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13" Jul 14 13:30:04.258760 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.258404 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ecb9c2a-202a-4cf8-be0d-82eb828138b5","Type":"ContainerDied","Data":"314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698"} Jul 14 13:30:04.258760 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.258428 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ecb9c2a-202a-4cf8-be0d-82eb828138b5","Type":"ContainerDied","Data":"80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4"} Jul 14 13:30:04.258760 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.258452 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1ecb9c2a-202a-4cf8-be0d-82eb828138b5","Type":"ContainerDied","Data":"a8d4a8625d687ad960e91fc45bde309c63e235ac98fa5b92479af7894366d3bd"} Jul 14 13:30:04.269271 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.269245 2583 scope.go:117] "RemoveContainer" containerID="5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1" Jul 14 13:30:04.276596 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.276571 2583 scope.go:117] "RemoveContainer" containerID="5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d" Jul 14 13:30:04.284025 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.284007 2583 scope.go:117] "RemoveContainer" containerID="e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8" Jul 14 13:30:04.290761 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.290732 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 13:30:04.291200 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.291177 2583 scope.go:117] "RemoveContainer" containerID="314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698" Jul 14 13:30:04.297500 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.297474 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 13:30:04.298258 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.298243 2583 scope.go:117] "RemoveContainer" containerID="80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4" Jul 14 13:30:04.305454 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.305436 2583 scope.go:117] "RemoveContainer" containerID="9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f" Jul 14 13:30:04.312299 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.312284 2583 scope.go:117] "RemoveContainer" containerID="bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13" Jul 14 13:30:04.312578 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:30:04.312553 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13\": container with ID starting with bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13 not found: ID does not exist" containerID="bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13" Jul 14 13:30:04.312665 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.312584 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13"} err="failed to get container status \"bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13\": rpc error: code = NotFound desc = could not find container \"bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13\": container with ID starting with bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13 not found: ID does not exist" Jul 14 13:30:04.312665 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.312621 2583 scope.go:117] "RemoveContainer" containerID="5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1" Jul 14 13:30:04.312869 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:30:04.312852 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1\": container with ID starting with 5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1 not found: ID does not exist" containerID="5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1" Jul 14 13:30:04.312909 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.312875 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1"} err="failed to get container status \"5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1\": rpc error: code = NotFound desc = could not find container \"5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1\": container with ID starting with 5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1 not found: ID does not exist" Jul 14 13:30:04.312909 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.312891 2583 scope.go:117] "RemoveContainer" containerID="5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d" Jul 14 13:30:04.313171 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:30:04.313149 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d\": container with ID starting with 5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d not found: ID does not exist" containerID="5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d" Jul 14 13:30:04.313228 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.313178 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d"} err="failed to get container status \"5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d\": rpc error: code = NotFound desc = could not find container \"5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d\": container with ID starting with 5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d not found: ID does not exist" Jul 14 13:30:04.313228 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.313198 2583 scope.go:117] "RemoveContainer" containerID="e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8" Jul 14 13:30:04.313439 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:30:04.313419 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8\": container with ID starting with e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8 not found: ID does not exist" containerID="e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8" Jul 14 13:30:04.313512 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.313443 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8"} err="failed to get container status \"e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8\": rpc error: code = NotFound desc = could not find container \"e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8\": container with ID starting with e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8 not found: ID does not exist" Jul 14 13:30:04.313512 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.313459 2583 scope.go:117] "RemoveContainer" containerID="314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698" Jul 14 13:30:04.313695 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:30:04.313681 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698\": container with ID starting with 314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698 not found: ID does not exist" containerID="314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698" Jul 14 13:30:04.313737 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.313700 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698"} err="failed to get container status \"314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698\": rpc error: code = NotFound desc = could not find container \"314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698\": container with ID starting with 314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698 not found: ID does not exist" Jul 14 13:30:04.313737 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.313712 2583 scope.go:117] "RemoveContainer" containerID="80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4" Jul 14 13:30:04.313932 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:30:04.313912 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4\": container with ID starting with 80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4 not found: ID does not exist" containerID="80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4" Jul 14 13:30:04.313978 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.313936 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4"} err="failed to get container status \"80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4\": rpc error: code = NotFound desc = could not find container \"80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4\": container with ID starting with 80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4 not found: ID does not exist" Jul 14 13:30:04.313978 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.313951 2583 scope.go:117] "RemoveContainer" containerID="9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f" Jul 14 13:30:04.314185 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:30:04.314172 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f\": container with ID starting with 9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f not found: ID does not exist" containerID="9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f" Jul 14 13:30:04.314235 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.314188 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f"} err="failed to get container status \"9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f\": rpc error: code = NotFound desc = could not find container \"9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f\": container with ID starting with 9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f not found: ID does not exist" Jul 14 13:30:04.314235 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.314201 2583 scope.go:117] "RemoveContainer" containerID="bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13" Jul 14 13:30:04.314422 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.314407 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13"} err="failed to get container status \"bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13\": rpc error: code = NotFound desc = could not find container \"bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13\": container with ID starting with bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13 not found: ID does not exist" Jul 14 13:30:04.314422 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.314422 2583 scope.go:117] "RemoveContainer" containerID="5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1" Jul 14 13:30:04.314636 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.314617 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1"} err="failed to get container status \"5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1\": rpc error: code = NotFound desc = could not find container \"5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1\": container with ID starting with 5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1 not found: ID does not exist" Jul 14 13:30:04.314704 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.314640 2583 scope.go:117] "RemoveContainer" containerID="5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d" Jul 14 13:30:04.315407 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.315018 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d"} err="failed to get container status \"5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d\": rpc error: code = NotFound desc = could not find container \"5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d\": container with ID starting with 5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d not found: ID does not exist" Jul 14 13:30:04.315407 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.315037 2583 scope.go:117] "RemoveContainer" containerID="e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8" Jul 14 13:30:04.315407 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.315307 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8"} err="failed to get container status \"e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8\": rpc error: code = NotFound desc = could not find container \"e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8\": container with ID starting with e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8 not found: ID does not exist" Jul 14 13:30:04.315407 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.315327 2583 scope.go:117] "RemoveContainer" containerID="314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698" Jul 14 13:30:04.317229 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.317201 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698"} err="failed to get container status \"314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698\": rpc error: code = NotFound desc = could not find container \"314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698\": container with ID starting with 314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698 not found: ID does not exist" Jul 14 13:30:04.317229 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.317231 2583 scope.go:117] "RemoveContainer" containerID="80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4" Jul 14 13:30:04.317683 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.317664 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4"} err="failed to get container status \"80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4\": rpc error: code = NotFound desc = could not find container \"80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4\": container with ID starting with 80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4 not found: ID does not exist" Jul 14 13:30:04.317755 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.317685 2583 scope.go:117] "RemoveContainer" containerID="9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f" Jul 14 13:30:04.317904 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.317880 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f"} err="failed to get container status \"9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f\": rpc error: code = NotFound desc = could not find container \"9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f\": container with ID starting with 9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f not found: ID does not exist" Jul 14 13:30:04.317997 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.317903 2583 scope.go:117] "RemoveContainer" containerID="bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13" Jul 14 13:30:04.318216 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.318197 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13"} err="failed to get container status \"bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13\": rpc error: code = NotFound desc = could not find container \"bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13\": container with ID starting with bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13 not found: ID does not exist" Jul 14 13:30:04.318279 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.318216 2583 scope.go:117] "RemoveContainer" containerID="5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1" Jul 14 13:30:04.318467 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.318448 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1"} err="failed to get container status \"5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1\": rpc error: code = NotFound desc = could not find container \"5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1\": container with ID starting with 5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1 not found: ID does not exist" Jul 14 13:30:04.318596 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.318469 2583 scope.go:117] "RemoveContainer" containerID="5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d" Jul 14 13:30:04.318684 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.318660 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d"} err="failed to get container status \"5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d\": rpc error: code = NotFound desc = could not find container \"5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d\": container with ID starting with 5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d not found: ID does not exist" Jul 14 13:30:04.318728 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.318687 2583 scope.go:117] "RemoveContainer" containerID="e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8" Jul 14 13:30:04.318893 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.318871 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8"} err="failed to get container status \"e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8\": rpc error: code = NotFound desc = could not find container \"e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8\": container with ID starting with e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8 not found: ID does not exist" Jul 14 13:30:04.318893 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.318891 2583 scope.go:117] "RemoveContainer" containerID="314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698" Jul 14 13:30:04.319134 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.319063 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698"} err="failed to get container status \"314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698\": rpc error: code = NotFound desc = could not find container \"314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698\": container with ID starting with 314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698 not found: ID does not exist" Jul 14 13:30:04.319210 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.319137 2583 scope.go:117] "RemoveContainer" containerID="80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4" Jul 14 13:30:04.319431 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.319407 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4"} err="failed to get container status \"80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4\": rpc error: code = NotFound desc = could not find container \"80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4\": container with ID starting with 80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4 not found: ID does not exist" Jul 14 13:30:04.319431 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.319428 2583 scope.go:117] "RemoveContainer" containerID="9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f" Jul 14 13:30:04.319663 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.319646 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f"} err="failed to get container status \"9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f\": rpc error: code = NotFound desc = could not find container \"9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f\": container with ID starting with 9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f not found: ID does not exist" Jul 14 13:30:04.319705 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.319664 2583 scope.go:117] "RemoveContainer" containerID="bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13" Jul 14 13:30:04.319872 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.319855 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13"} err="failed to get container status \"bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13\": rpc error: code = NotFound desc = could not find container \"bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13\": container with ID starting with bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13 not found: ID does not exist" Jul 14 13:30:04.319922 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.319872 2583 scope.go:117] "RemoveContainer" containerID="5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1" Jul 14 13:30:04.320071 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.320056 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1"} err="failed to get container status \"5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1\": rpc error: code = NotFound desc = could not find container \"5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1\": container with ID starting with 5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1 not found: ID does not exist" Jul 14 13:30:04.320141 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.320071 2583 scope.go:117] "RemoveContainer" containerID="5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d" Jul 14 13:30:04.320326 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.320301 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d"} err="failed to get container status \"5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d\": rpc error: code = NotFound desc = could not find container \"5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d\": container with ID starting with 5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d not found: ID does not exist" Jul 14 13:30:04.320326 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.320320 2583 scope.go:117] "RemoveContainer" containerID="e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8" Jul 14 13:30:04.320543 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.320525 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8"} err="failed to get container status \"e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8\": rpc error: code = NotFound desc = could not find container \"e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8\": container with ID starting with e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8 not found: ID does not exist" Jul 14 13:30:04.320591 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.320543 2583 scope.go:117] "RemoveContainer" containerID="314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698" Jul 14 13:30:04.320762 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.320745 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698"} err="failed to get container status \"314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698\": rpc error: code = NotFound desc = could not find container \"314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698\": container with ID starting with 314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698 not found: ID does not exist" Jul 14 13:30:04.320805 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.320763 2583 scope.go:117] "RemoveContainer" containerID="80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4" Jul 14 13:30:04.320988 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.320972 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4"} err="failed to get container status \"80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4\": rpc error: code = NotFound desc = could not find container \"80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4\": container with ID starting with 80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4 not found: ID does not exist" Jul 14 13:30:04.321037 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.320989 2583 scope.go:117] "RemoveContainer" containerID="9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f" Jul 14 13:30:04.321207 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.321190 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f"} err="failed to get container status \"9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f\": rpc error: code = NotFound desc = could not find container \"9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f\": container with ID starting with 9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f not found: ID does not exist" Jul 14 13:30:04.321271 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.321207 2583 scope.go:117] "RemoveContainer" containerID="bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13" Jul 14 13:30:04.321400 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.321384 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13"} err="failed to get container status \"bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13\": rpc error: code = NotFound desc = could not find container \"bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13\": container with ID starting with bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13 not found: ID does not exist" Jul 14 13:30:04.321440 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.321400 2583 scope.go:117] "RemoveContainer" containerID="5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1" Jul 14 13:30:04.321586 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.321567 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1"} err="failed to get container status \"5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1\": rpc error: code = NotFound desc = could not find container \"5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1\": container with ID starting with 5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1 not found: ID does not exist" Jul 14 13:30:04.321624 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.321587 2583 scope.go:117] "RemoveContainer" containerID="5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d" Jul 14 13:30:04.321823 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.321804 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d"} err="failed to get container status \"5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d\": rpc error: code = NotFound desc = could not find container \"5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d\": container with ID starting with 5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d not found: ID does not exist" Jul 14 13:30:04.321877 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.321823 2583 scope.go:117] "RemoveContainer" containerID="e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8" Jul 14 13:30:04.322031 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.322016 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8"} err="failed to get container status \"e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8\": rpc error: code = NotFound desc = could not find container \"e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8\": container with ID starting with e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8 not found: ID does not exist" Jul 14 13:30:04.322097 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.322031 2583 scope.go:117] "RemoveContainer" containerID="314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698" Jul 14 13:30:04.322257 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.322238 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698"} err="failed to get container status \"314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698\": rpc error: code = NotFound desc = could not find container \"314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698\": container with ID starting with 314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698 not found: ID does not exist" Jul 14 13:30:04.322304 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.322258 2583 scope.go:117] "RemoveContainer" containerID="80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4" Jul 14 13:30:04.322439 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.322421 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4"} err="failed to get container status \"80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4\": rpc error: code = NotFound desc = could not find container \"80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4\": container with ID starting with 80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4 not found: ID does not exist" Jul 14 13:30:04.322481 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.322438 2583 scope.go:117] "RemoveContainer" containerID="9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f" Jul 14 13:30:04.322646 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.322629 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f"} err="failed to get container status \"9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f\": rpc error: code = NotFound desc = could not find container \"9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f\": container with ID starting with 9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f not found: ID does not exist" Jul 14 13:30:04.322686 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.322646 2583 scope.go:117] "RemoveContainer" containerID="bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13" Jul 14 13:30:04.322845 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.322830 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13"} err="failed to get container status \"bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13\": rpc error: code = NotFound desc = could not find container \"bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13\": container with ID starting with bd69bf60b64744b25bd4df3e1f7cee75f9b7bda5cf7e506f155012e36b79ae13 not found: ID does not exist" Jul 14 13:30:04.322845 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.322843 2583 scope.go:117] "RemoveContainer" containerID="5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1" Jul 14 13:30:04.323035 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.323019 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1"} err="failed to get container status \"5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1\": rpc error: code = NotFound desc = could not find container \"5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1\": container with ID starting with 5a0f37f498c2e43b5d317317e8da43e0e396a773782870b8d7abbabb200d48b1 not found: ID does not exist" Jul 14 13:30:04.323097 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.323036 2583 scope.go:117] "RemoveContainer" containerID="5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d" Jul 14 13:30:04.323241 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.323226 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d"} err="failed to get container status \"5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d\": rpc error: code = NotFound desc = could not find container \"5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d\": container with ID starting with 5fb2e3faeb3faf9739b085f6b48f3b765d1f18dbab168db214a46e62a63feb0d not found: ID does not exist" Jul 14 13:30:04.323286 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.323241 2583 scope.go:117] "RemoveContainer" containerID="e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8" Jul 14 13:30:04.323411 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.323397 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8"} err="failed to get container status \"e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8\": rpc error: code = NotFound desc = could not find container \"e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8\": container with ID starting with e424b38e9b65d76fa7aed0866bcc4b6638a7bf8429025c73e0cd2db5bd4d54f8 not found: ID does not exist" Jul 14 13:30:04.323448 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.323411 2583 scope.go:117] "RemoveContainer" containerID="314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698" Jul 14 13:30:04.323607 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.323590 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698"} err="failed to get container status \"314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698\": rpc error: code = NotFound desc = could not find container \"314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698\": container with ID starting with 314467914cb13aef9e146217469353c92c22a04b3eb9e26ab57553e39dfcf698 not found: ID does not exist" Jul 14 13:30:04.323649 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.323607 2583 scope.go:117] "RemoveContainer" containerID="80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4" Jul 14 13:30:04.323785 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.323770 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4"} err="failed to get container status \"80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4\": rpc error: code = NotFound desc = could not find container \"80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4\": container with ID starting with 80520e71a59608ff251290f00acbd1e4fc62077cf61906fe47b6cccd8f5c67f4 not found: ID does not exist" Jul 14 13:30:04.323785 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.323784 2583 scope.go:117] "RemoveContainer" containerID="9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f" Jul 14 13:30:04.323976 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.323959 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f"} err="failed to get container status \"9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f\": rpc error: code = NotFound desc = could not find container \"9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f\": container with ID starting with 9e3e53897defb6256a62650d3bb164d4d836b2cc8417318166fba40627906a1f not found: ID does not exist" Jul 14 13:30:04.335144 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.335109 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 13:30:04.335388 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.335376 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="config-reloader" Jul 14 13:30:04.335429 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.335389 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="config-reloader" Jul 14 13:30:04.335429 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.335404 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="prometheus" Jul 14 13:30:04.335429 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.335410 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="prometheus" Jul 14 13:30:04.335515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.335439 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="kube-rbac-proxy-thanos" Jul 14 13:30:04.335515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.335445 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="kube-rbac-proxy-thanos" Jul 14 13:30:04.335515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.335452 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="thanos-sidecar" Jul 14 13:30:04.335515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.335458 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="thanos-sidecar" Jul 14 13:30:04.335515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.335467 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="kube-rbac-proxy" Jul 14 13:30:04.335515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.335472 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="kube-rbac-proxy" Jul 14 13:30:04.335515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.335478 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="init-config-reloader" Jul 14 13:30:04.335515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.335483 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="init-config-reloader" Jul 14 13:30:04.335515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.335489 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="kube-rbac-proxy-web" Jul 14 13:30:04.335515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.335494 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="kube-rbac-proxy-web" Jul 14 13:30:04.335790 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.335531 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="thanos-sidecar" Jul 14 13:30:04.335790 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.335540 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="config-reloader" Jul 14 13:30:04.335790 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.335546 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="prometheus" Jul 14 13:30:04.335790 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.335552 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="kube-rbac-proxy-web" Jul 14 13:30:04.335790 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.335557 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="kube-rbac-proxy" Jul 14 13:30:04.335790 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.335563 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" containerName="kube-rbac-proxy-thanos" Jul 14 13:30:04.340062 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.340045 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.343052 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.343027 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 14 13:30:04.343052 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.343042 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 14 13:30:04.343257 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.343122 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 14 13:30:04.343257 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.343155 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 14 13:30:04.343372 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.343357 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 14 13:30:04.343516 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.343502 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 14 13:30:04.343700 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.343687 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 14 13:30:04.343914 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.343888 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bhlr2\"" Jul 14 13:30:04.343914 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.343901 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 14 13:30:04.344039 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.343930 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5agnm3a4tti59\"" Jul 14 13:30:04.344274 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344258 2583 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-web-config\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:30:04.344318 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344278 2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:30:04.344318 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344289 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:30:04.344318 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344298 2583 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-configmap-metrics-client-ca\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:30:04.344318 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344308 2583 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-kube-rbac-proxy\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:30:04.344318 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344316 2583 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-config\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:30:04.344544 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344325 2583 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:30:04.344544 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344335 2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:30:04.344544 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344345 2583 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:30:04.344544 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344354 2583 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-grpc-tls\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:30:04.344544 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344362 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcqhr\" (UniqueName: \"kubernetes.io/projected/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-kube-api-access-bcqhr\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:30:04.344544 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344370 2583 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:30:04.344544 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344379 2583 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:30:04.344544 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344390 2583 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-config-out\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:30:04.344544 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344403 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:30:04.344544 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344415 2583 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-prometheus-k8s-db\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:30:04.344544 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344426 2583 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-tls-assets\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:30:04.344544 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344439 2583 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1ecb9c2a-202a-4cf8-be0d-82eb828138b5-secret-metrics-client-certs\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:30:04.344544 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344479 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 14 13:30:04.344544 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344506 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 14 13:30:04.344544 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344505 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 14 13:30:04.345046 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.344616 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 14 13:30:04.345810 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.345795 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 14 13:30:04.348017 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.347997 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 14 13:30:04.351211 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.351195 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 14 13:30:04.359474 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.359451 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 13:30:04.444989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.444951 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-config\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.445243 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.445004 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400f61c2-6d7a-4727-a3ca-350ddfed4218-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.445243 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.445049 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.445243 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.445069 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400f61c2-6d7a-4727-a3ca-350ddfed4218-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.445243 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.445123 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/400f61c2-6d7a-4727-a3ca-350ddfed4218-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.445243 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.445144 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.445243 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.445162 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/400f61c2-6d7a-4727-a3ca-350ddfed4218-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.445243 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.445211 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/400f61c2-6d7a-4727-a3ca-350ddfed4218-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.445243 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.445244 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.445515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.445278 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/400f61c2-6d7a-4727-a3ca-350ddfed4218-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.445515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.445300 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/400f61c2-6d7a-4727-a3ca-350ddfed4218-config-out\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.445515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.445322 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.445515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.445365 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.445515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.445386 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wtv8\" (UniqueName: \"kubernetes.io/projected/400f61c2-6d7a-4727-a3ca-350ddfed4218-kube-api-access-7wtv8\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.445515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.445411 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/400f61c2-6d7a-4727-a3ca-350ddfed4218-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.445515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.445444 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.445515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.445464 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/400f61c2-6d7a-4727-a3ca-350ddfed4218-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.445515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.445489 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.445515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.445510 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400f61c2-6d7a-4727-a3ca-350ddfed4218-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.445818 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.445542 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-web-config\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.491255 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.491224 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ecb9c2a-202a-4cf8-be0d-82eb828138b5" path="/var/lib/kubelet/pods/1ecb9c2a-202a-4cf8-be0d-82eb828138b5/volumes" Jul 14 13:30:04.546968 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.546936 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.547132 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.546973 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wtv8\" (UniqueName: \"kubernetes.io/projected/400f61c2-6d7a-4727-a3ca-350ddfed4218-kube-api-access-7wtv8\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.547132 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.546994 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/400f61c2-6d7a-4727-a3ca-350ddfed4218-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.547132 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.547016 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.547132 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.547036 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/400f61c2-6d7a-4727-a3ca-350ddfed4218-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.547132 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.547058 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.547132 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.547107 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400f61c2-6d7a-4727-a3ca-350ddfed4218-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.547439 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.547137 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-web-config\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.547439 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.547162 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-config\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.547439 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.547199 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400f61c2-6d7a-4727-a3ca-350ddfed4218-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.547439 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.547233 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.547439 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.547267 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400f61c2-6d7a-4727-a3ca-350ddfed4218-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.547439 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.547292 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/400f61c2-6d7a-4727-a3ca-350ddfed4218-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.547439 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.547313 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.547770 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.547615 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/400f61c2-6d7a-4727-a3ca-350ddfed4218-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.547770 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.547656 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/400f61c2-6d7a-4727-a3ca-350ddfed4218-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.547770 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.547690 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.547770 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.547723 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/400f61c2-6d7a-4727-a3ca-350ddfed4218-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.547770 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.547754 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/400f61c2-6d7a-4727-a3ca-350ddfed4218-config-out\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.548001 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.547796 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.548001 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.547911 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/400f61c2-6d7a-4727-a3ca-350ddfed4218-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.548118 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.548021 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400f61c2-6d7a-4727-a3ca-350ddfed4218-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.548538 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.548321 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400f61c2-6d7a-4727-a3ca-350ddfed4218-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.548654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.548598 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/400f61c2-6d7a-4727-a3ca-350ddfed4218-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.550524 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.548742 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/400f61c2-6d7a-4727-a3ca-350ddfed4218-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.550638 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.550622 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.551431 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.550915 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/400f61c2-6d7a-4727-a3ca-350ddfed4218-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.551431 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.551120 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400f61c2-6d7a-4727-a3ca-350ddfed4218-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.551431 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.551377 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.551431 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.551384 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.551431 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.551385 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.551431 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.551427 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-web-config\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.551805 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.551735 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/400f61c2-6d7a-4727-a3ca-350ddfed4218-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.552216 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.552192 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.552474 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.552453 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-config\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.552669 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.552651 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/400f61c2-6d7a-4727-a3ca-350ddfed4218-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.552911 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.552893 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.553124 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.553107 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/400f61c2-6d7a-4727-a3ca-350ddfed4218-config-out\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.553457 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.553439 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/400f61c2-6d7a-4727-a3ca-350ddfed4218-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.557495 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.557478 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wtv8\" (UniqueName: \"kubernetes.io/projected/400f61c2-6d7a-4727-a3ca-350ddfed4218-kube-api-access-7wtv8\") pod \"prometheus-k8s-0\" (UID: \"400f61c2-6d7a-4727-a3ca-350ddfed4218\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.649525 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.649495 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:04.791688 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:04.791657 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 13:30:04.794893 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:30:04.794865 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod400f61c2_6d7a_4727_a3ca_350ddfed4218.slice/crio-369c2086d64a1ab0a064f5920d97cfad41c776c5c81cab5907293680ac5b8a39 WatchSource:0}: Error finding container 369c2086d64a1ab0a064f5920d97cfad41c776c5c81cab5907293680ac5b8a39: Status 404 returned error can't find the container with id 369c2086d64a1ab0a064f5920d97cfad41c776c5c81cab5907293680ac5b8a39 Jul 14 13:30:05.264287 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:05.264253 2583 generic.go:356] "Generic (PLEG): container finished" podID="400f61c2-6d7a-4727-a3ca-350ddfed4218" containerID="77ab3cc8ff1fe55ab480a392b913268da549c613bfabce318eb1948a0c1fa9ce" exitCode=0 Jul 14 13:30:05.264752 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:05.264341 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400f61c2-6d7a-4727-a3ca-350ddfed4218","Type":"ContainerDied","Data":"77ab3cc8ff1fe55ab480a392b913268da549c613bfabce318eb1948a0c1fa9ce"} Jul 14 13:30:05.264752 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:05.264429 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400f61c2-6d7a-4727-a3ca-350ddfed4218","Type":"ContainerStarted","Data":"369c2086d64a1ab0a064f5920d97cfad41c776c5c81cab5907293680ac5b8a39"} Jul 14 13:30:06.270741 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:06.270704 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400f61c2-6d7a-4727-a3ca-350ddfed4218","Type":"ContainerStarted","Data":"ff7e5fab9a0e0b185ef918f3d8d263402ae39d609122de80b69778958ab24a6a"} Jul 14 13:30:06.271122 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:06.270793 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400f61c2-6d7a-4727-a3ca-350ddfed4218","Type":"ContainerStarted","Data":"2266c51906fbe4f229730c86dcfab3c62b0543abdd3bf85dccc3f866228fe97d"} Jul 14 13:30:06.271122 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:06.270820 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400f61c2-6d7a-4727-a3ca-350ddfed4218","Type":"ContainerStarted","Data":"2c50bbea8914a9610e1b4c2fa9e7e6c36f0fc17c8896eeb87b385d74cd00ee27"} Jul 14 13:30:06.271122 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:06.270844 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400f61c2-6d7a-4727-a3ca-350ddfed4218","Type":"ContainerStarted","Data":"363d8495ebf7c60553b356975b4818984eb87761e9f3b0e0c551136754c80a5b"} Jul 14 13:30:06.271122 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:06.270867 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400f61c2-6d7a-4727-a3ca-350ddfed4218","Type":"ContainerStarted","Data":"9649d5477cc51dd711a6994ac9eb37f0af543d8d7a0a55a000fa0c04e9df0177"} Jul 14 13:30:06.271122 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:06.270890 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400f61c2-6d7a-4727-a3ca-350ddfed4218","Type":"ContainerStarted","Data":"f71e50e904a6f061e5e23af50acc75fcaa3300461f086c391e5b0678f0fc4440"} Jul 14 13:30:06.305715 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:06.305654 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.28392669 podStartE2EDuration="2.305637533s" totalImagesPullingTime="0s" totalInitContainerRuntime="21.710843ms" isStatefulPod=false podCreationTimestamp="2026-07-14 13:30:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:30:06.303308533 +0000 UTC m=+169.830459172" watchObservedRunningTime="2026-07-14 13:30:06.305637533 +0000 UTC m=+169.832788170" Jul 14 13:30:07.329986 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:07.329951 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:30:07.611717 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:07.611621 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:30:08.212136 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:08.212074 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:30:08.546225 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:08.546188 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:30:08.649103 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:08.649029 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:30:08.953124 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:08.952999 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:30:09.024674 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:09.024640 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:30:09.649975 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:09.649919 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:30:14.354540 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:14.354502 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:30:14.442928 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:14.442886 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:30:19.243732 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:19.243693 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:30:19.394486 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:19.394447 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:30:40.597145 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:40.597099 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:30:40.673644 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:40.673610 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:30:57.630678 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:57.630626 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:30:57.731872 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:30:57.731833 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:31:04.649863 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:04.649824 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:31:04.665057 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:04.665029 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:31:05.492915 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:05.492884 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 13:31:24.484665 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:24.484625 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:31:25.236474 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:25.236427 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:31:28.154786 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:28.154729 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:31:28.250422 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:28.250385 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:31:29.010348 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:29.010305 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:31:29.110466 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:29.110435 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:31:29.663597 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:29.663568 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:31:29.778098 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:29.778049 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:31:31.444099 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:31.444049 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:31:31.843432 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:31.843394 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:31:34.856578 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:34.856534 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:31:34.923891 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:34.923857 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:31:39.374128 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:39.374022 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:31:39.649608 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:39.649504 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:31:40.306171 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:40.306132 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:31:40.405304 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:40.405270 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:31:46.408806 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:46.408775 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:31:46.522035 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:46.522002 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:31:48.414764 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:48.414722 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:31:59.928167 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:31:59.928128 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:00.031338 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:00.031303 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:00.470580 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:00.470544 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:00.564067 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:00.564034 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:01.092415 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.092377 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:01.285743 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.285702 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:01.385516 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.385420 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:01.449354 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.449326 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6"] Jul 14 13:32:01.452279 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.452262 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" Jul 14 13:32:01.455728 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.455706 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 14 13:32:01.456194 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.456177 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 14 13:32:01.456254 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.456217 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 14 13:32:01.456305 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.456258 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-b48w9\"" Jul 14 13:32:01.456899 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.456884 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 14 13:32:01.474928 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.471674 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6"] Jul 14 13:32:01.546452 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.546410 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrgrp\" (UniqueName: \"kubernetes.io/projected/20c64d8b-5726-4930-8be6-02866c1b10ca-kube-api-access-mrgrp\") pod \"opendatahub-operator-controller-manager-9464f545f-xqxn6\" (UID: \"20c64d8b-5726-4930-8be6-02866c1b10ca\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" Jul 14 13:32:01.546645 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.546464 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20c64d8b-5726-4930-8be6-02866c1b10ca-webhook-cert\") pod \"opendatahub-operator-controller-manager-9464f545f-xqxn6\" (UID: \"20c64d8b-5726-4930-8be6-02866c1b10ca\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" Jul 14 13:32:01.546645 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.546498 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20c64d8b-5726-4930-8be6-02866c1b10ca-tmp\") pod \"opendatahub-operator-controller-manager-9464f545f-xqxn6\" (UID: \"20c64d8b-5726-4930-8be6-02866c1b10ca\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" Jul 14 13:32:01.546645 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.546539 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/20c64d8b-5726-4930-8be6-02866c1b10ca-manifests\") pod \"opendatahub-operator-controller-manager-9464f545f-xqxn6\" (UID: \"20c64d8b-5726-4930-8be6-02866c1b10ca\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" Jul 14 13:32:01.546645 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.546632 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/20c64d8b-5726-4930-8be6-02866c1b10ca-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9464f545f-xqxn6\" (UID: \"20c64d8b-5726-4930-8be6-02866c1b10ca\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" Jul 14 13:32:01.580577 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.580538 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:01.647843 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.647755 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/20c64d8b-5726-4930-8be6-02866c1b10ca-manifests\") pod \"opendatahub-operator-controller-manager-9464f545f-xqxn6\" (UID: \"20c64d8b-5726-4930-8be6-02866c1b10ca\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" Jul 14 13:32:01.647843 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.647813 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/20c64d8b-5726-4930-8be6-02866c1b10ca-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9464f545f-xqxn6\" (UID: \"20c64d8b-5726-4930-8be6-02866c1b10ca\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" Jul 14 13:32:01.648045 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.647860 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrgrp\" (UniqueName: \"kubernetes.io/projected/20c64d8b-5726-4930-8be6-02866c1b10ca-kube-api-access-mrgrp\") pod \"opendatahub-operator-controller-manager-9464f545f-xqxn6\" (UID: \"20c64d8b-5726-4930-8be6-02866c1b10ca\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" Jul 14 13:32:01.648045 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.647892 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20c64d8b-5726-4930-8be6-02866c1b10ca-webhook-cert\") pod \"opendatahub-operator-controller-manager-9464f545f-xqxn6\" (UID: \"20c64d8b-5726-4930-8be6-02866c1b10ca\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" Jul 14 13:32:01.648045 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.647923 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20c64d8b-5726-4930-8be6-02866c1b10ca-tmp\") pod \"opendatahub-operator-controller-manager-9464f545f-xqxn6\" (UID: \"20c64d8b-5726-4930-8be6-02866c1b10ca\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" Jul 14 13:32:01.648223 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.648195 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/20c64d8b-5726-4930-8be6-02866c1b10ca-manifests\") pod \"opendatahub-operator-controller-manager-9464f545f-xqxn6\" (UID: \"20c64d8b-5726-4930-8be6-02866c1b10ca\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" Jul 14 13:32:01.650201 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.650173 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20c64d8b-5726-4930-8be6-02866c1b10ca-tmp\") pod \"opendatahub-operator-controller-manager-9464f545f-xqxn6\" (UID: \"20c64d8b-5726-4930-8be6-02866c1b10ca\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" Jul 14 13:32:01.650336 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.650319 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/20c64d8b-5726-4930-8be6-02866c1b10ca-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9464f545f-xqxn6\" (UID: \"20c64d8b-5726-4930-8be6-02866c1b10ca\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" Jul 14 13:32:01.650451 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.650432 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20c64d8b-5726-4930-8be6-02866c1b10ca-webhook-cert\") pod \"opendatahub-operator-controller-manager-9464f545f-xqxn6\" (UID: \"20c64d8b-5726-4930-8be6-02866c1b10ca\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" Jul 14 13:32:01.670766 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.670737 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrgrp\" (UniqueName: \"kubernetes.io/projected/20c64d8b-5726-4930-8be6-02866c1b10ca-kube-api-access-mrgrp\") pod \"opendatahub-operator-controller-manager-9464f545f-xqxn6\" (UID: \"20c64d8b-5726-4930-8be6-02866c1b10ca\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" Jul 14 13:32:01.691566 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.691533 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:01.763006 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.762959 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" Jul 14 13:32:01.795767 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.795731 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:01.887832 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.887783 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:01.928391 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.928313 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6"] Jul 14 13:32:01.932117 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:32:01.932089 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20c64d8b_5726_4930_8be6_02866c1b10ca.slice/crio-46a6886afc93c24c5753c1e5377bcf438a69a62bdb1e75cad63cd6044dfb968e WatchSource:0}: Error finding container 46a6886afc93c24c5753c1e5377bcf438a69a62bdb1e75cad63cd6044dfb968e: Status 404 returned error can't find the container with id 46a6886afc93c24c5753c1e5377bcf438a69a62bdb1e75cad63cd6044dfb968e Jul 14 13:32:01.996201 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:01.996167 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:02.098138 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:02.098101 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:02.213631 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:02.213539 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:02.677350 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:02.677302 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" event={"ID":"20c64d8b-5726-4930-8be6-02866c1b10ca","Type":"ContainerStarted","Data":"46a6886afc93c24c5753c1e5377bcf438a69a62bdb1e75cad63cd6044dfb968e"} Jul 14 13:32:05.691008 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:05.690959 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" event={"ID":"20c64d8b-5726-4930-8be6-02866c1b10ca","Type":"ContainerStarted","Data":"09b994e29db2f1d11e922a09c34990e03eab94e314aad9faaa8dfec6c46d7662"} Jul 14 13:32:06.696129 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:06.696096 2583 generic.go:356] "Generic (PLEG): container finished" podID="20c64d8b-5726-4930-8be6-02866c1b10ca" containerID="09b994e29db2f1d11e922a09c34990e03eab94e314aad9faaa8dfec6c46d7662" exitCode=0 Jul 14 13:32:06.696579 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:06.696175 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" event={"ID":"20c64d8b-5726-4930-8be6-02866c1b10ca","Type":"ContainerDied","Data":"09b994e29db2f1d11e922a09c34990e03eab94e314aad9faaa8dfec6c46d7662"} Jul 14 13:32:07.701713 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:07.701670 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" event={"ID":"20c64d8b-5726-4930-8be6-02866c1b10ca","Type":"ContainerStarted","Data":"b41f20335e86b8467d450fd9d72acec37fd214a202143f11ee8a6683f454a454"} Jul 14 13:32:07.702141 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:07.701803 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" Jul 14 13:32:12.682669 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:12.682633 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:12.820214 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:12.820175 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:18.325250 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:18.325219 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 13:32:18.329685 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:18.329653 2583 kubelet.go:1648] "Image garbage collection succeeded" Jul 14 13:32:18.331304 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:18.331282 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 13:32:18.707893 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:18.707811 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" Jul 14 13:32:18.736195 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:18.736143 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-xqxn6" podStartSLOduration=13.273185298 podStartE2EDuration="17.736127413s" totalImagesPullingTime="3.453034856s" totalInitContainerRuntime="1.009907259s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:32:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:32:07.756760988 +0000 UTC m=+291.283911624" watchObservedRunningTime="2026-07-14 13:32:18.736127413 +0000 UTC m=+302.263278049" Jul 14 13:32:20.577019 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:20.576985 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:21.560221 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:21.560188 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-594bccb575-nmtvq"] Jul 14 13:32:21.562350 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:21.562331 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-594bccb575-nmtvq" Jul 14 13:32:21.576241 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:21.576217 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 14 13:32:21.576672 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:21.576656 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 14 13:32:21.576969 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:21.576951 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 14 13:32:21.577059 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:21.577035 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 14 13:32:21.577377 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:21.577181 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xtqjz\"" Jul 14 13:32:21.584802 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:21.584772 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-594bccb575-nmtvq"] Jul 14 13:32:21.719147 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:21.719106 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/118356c5-d6ff-4662-b421-c2066cedec88-tls-certs\") pod \"kube-auth-proxy-594bccb575-nmtvq\" (UID: \"118356c5-d6ff-4662-b421-c2066cedec88\") " pod="openshift-ingress/kube-auth-proxy-594bccb575-nmtvq" Jul 14 13:32:21.719147 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:21.719151 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/118356c5-d6ff-4662-b421-c2066cedec88-tmp\") pod \"kube-auth-proxy-594bccb575-nmtvq\" (UID: \"118356c5-d6ff-4662-b421-c2066cedec88\") " pod="openshift-ingress/kube-auth-proxy-594bccb575-nmtvq" Jul 14 13:32:21.719379 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:21.719204 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42gpx\" (UniqueName: \"kubernetes.io/projected/118356c5-d6ff-4662-b421-c2066cedec88-kube-api-access-42gpx\") pod \"kube-auth-proxy-594bccb575-nmtvq\" (UID: \"118356c5-d6ff-4662-b421-c2066cedec88\") " pod="openshift-ingress/kube-auth-proxy-594bccb575-nmtvq" Jul 14 13:32:21.820618 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:21.820521 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/118356c5-d6ff-4662-b421-c2066cedec88-tls-certs\") pod \"kube-auth-proxy-594bccb575-nmtvq\" (UID: \"118356c5-d6ff-4662-b421-c2066cedec88\") " pod="openshift-ingress/kube-auth-proxy-594bccb575-nmtvq" Jul 14 13:32:21.820618 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:21.820560 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/118356c5-d6ff-4662-b421-c2066cedec88-tmp\") pod \"kube-auth-proxy-594bccb575-nmtvq\" (UID: \"118356c5-d6ff-4662-b421-c2066cedec88\") " pod="openshift-ingress/kube-auth-proxy-594bccb575-nmtvq" Jul 14 13:32:21.820618 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:21.820593 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42gpx\" (UniqueName: \"kubernetes.io/projected/118356c5-d6ff-4662-b421-c2066cedec88-kube-api-access-42gpx\") pod \"kube-auth-proxy-594bccb575-nmtvq\" (UID: \"118356c5-d6ff-4662-b421-c2066cedec88\") " pod="openshift-ingress/kube-auth-proxy-594bccb575-nmtvq" Jul 14 13:32:21.823000 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:21.822960 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/118356c5-d6ff-4662-b421-c2066cedec88-tmp\") pod \"kube-auth-proxy-594bccb575-nmtvq\" (UID: \"118356c5-d6ff-4662-b421-c2066cedec88\") " pod="openshift-ingress/kube-auth-proxy-594bccb575-nmtvq" Jul 14 13:32:21.823200 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:21.823175 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/118356c5-d6ff-4662-b421-c2066cedec88-tls-certs\") pod \"kube-auth-proxy-594bccb575-nmtvq\" (UID: \"118356c5-d6ff-4662-b421-c2066cedec88\") " pod="openshift-ingress/kube-auth-proxy-594bccb575-nmtvq" Jul 14 13:32:21.839984 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:21.839949 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42gpx\" (UniqueName: \"kubernetes.io/projected/118356c5-d6ff-4662-b421-c2066cedec88-kube-api-access-42gpx\") pod \"kube-auth-proxy-594bccb575-nmtvq\" (UID: \"118356c5-d6ff-4662-b421-c2066cedec88\") " pod="openshift-ingress/kube-auth-proxy-594bccb575-nmtvq" Jul 14 13:32:21.871939 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:21.871898 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-594bccb575-nmtvq" Jul 14 13:32:22.039267 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:22.039230 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-594bccb575-nmtvq"] Jul 14 13:32:22.045768 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:32:22.045739 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod118356c5_d6ff_4662_b421_c2066cedec88.slice/crio-c38ef7880fabb5d89967dd40598add736fcb4e73cb7814c2c39df6899b0ef5d7 WatchSource:0}: Error finding container c38ef7880fabb5d89967dd40598add736fcb4e73cb7814c2c39df6899b0ef5d7: Status 404 returned error can't find the container with id c38ef7880fabb5d89967dd40598add736fcb4e73cb7814c2c39df6899b0ef5d7 Jul 14 13:32:22.047608 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:22.047593 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 13:32:22.090659 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:22.090580 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jx5df"] Jul 14 13:32:22.093305 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:22.093288 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-jx5df" Jul 14 13:32:22.096289 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:22.096267 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 14 13:32:22.096510 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:22.096494 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-p72lh\"" Jul 14 13:32:22.110852 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:22.110819 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jx5df"] Jul 14 13:32:22.127275 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:22.127244 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:22.223767 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:22.223735 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/218fb2da-1c3e-4b3b-984a-0861060202a3-cert\") pod \"odh-model-controller-7998c67dc4-jx5df\" (UID: \"218fb2da-1c3e-4b3b-984a-0861060202a3\") " pod="opendatahub/odh-model-controller-7998c67dc4-jx5df" Jul 14 13:32:22.223952 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:22.223794 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkv8x\" (UniqueName: \"kubernetes.io/projected/218fb2da-1c3e-4b3b-984a-0861060202a3-kube-api-access-zkv8x\") pod \"odh-model-controller-7998c67dc4-jx5df\" (UID: \"218fb2da-1c3e-4b3b-984a-0861060202a3\") " pod="opendatahub/odh-model-controller-7998c67dc4-jx5df" Jul 14 13:32:22.227696 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:22.227666 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:22.321568 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:22.321533 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:22.324620 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:22.324598 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/218fb2da-1c3e-4b3b-984a-0861060202a3-cert\") pod \"odh-model-controller-7998c67dc4-jx5df\" (UID: \"218fb2da-1c3e-4b3b-984a-0861060202a3\") " pod="opendatahub/odh-model-controller-7998c67dc4-jx5df" Jul 14 13:32:22.324706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:22.324657 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkv8x\" (UniqueName: \"kubernetes.io/projected/218fb2da-1c3e-4b3b-984a-0861060202a3-kube-api-access-zkv8x\") pod \"odh-model-controller-7998c67dc4-jx5df\" (UID: \"218fb2da-1c3e-4b3b-984a-0861060202a3\") " pod="opendatahub/odh-model-controller-7998c67dc4-jx5df" Jul 14 13:32:22.324768 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:32:22.324754 2583 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 14 13:32:22.324841 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:32:22.324828 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/218fb2da-1c3e-4b3b-984a-0861060202a3-cert podName:218fb2da-1c3e-4b3b-984a-0861060202a3 nodeName:}" failed. No retries permitted until 2026-07-14 13:32:22.824808926 +0000 UTC m=+306.351959547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/218fb2da-1c3e-4b3b-984a-0861060202a3-cert") pod "odh-model-controller-7998c67dc4-jx5df" (UID: "218fb2da-1c3e-4b3b-984a-0861060202a3") : secret "odh-model-controller-webhook-cert" not found Jul 14 13:32:22.341136 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:22.341051 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkv8x\" (UniqueName: \"kubernetes.io/projected/218fb2da-1c3e-4b3b-984a-0861060202a3-kube-api-access-zkv8x\") pod \"odh-model-controller-7998c67dc4-jx5df\" (UID: \"218fb2da-1c3e-4b3b-984a-0861060202a3\") " pod="opendatahub/odh-model-controller-7998c67dc4-jx5df" Jul 14 13:32:22.551709 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:22.551670 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:22.757656 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:22.757621 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-594bccb575-nmtvq" event={"ID":"118356c5-d6ff-4662-b421-c2066cedec88","Type":"ContainerStarted","Data":"c38ef7880fabb5d89967dd40598add736fcb4e73cb7814c2c39df6899b0ef5d7"} Jul 14 13:32:22.829450 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:22.829417 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/218fb2da-1c3e-4b3b-984a-0861060202a3-cert\") pod \"odh-model-controller-7998c67dc4-jx5df\" (UID: \"218fb2da-1c3e-4b3b-984a-0861060202a3\") " pod="opendatahub/odh-model-controller-7998c67dc4-jx5df" Jul 14 13:32:22.831939 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:22.831902 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/218fb2da-1c3e-4b3b-984a-0861060202a3-cert\") pod \"odh-model-controller-7998c67dc4-jx5df\" (UID: \"218fb2da-1c3e-4b3b-984a-0861060202a3\") " pod="opendatahub/odh-model-controller-7998c67dc4-jx5df" Jul 14 13:32:23.005292 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.005248 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-jx5df" Jul 14 13:32:23.181556 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.181521 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:23.194234 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.194193 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jx5df"] Jul 14 13:32:23.194395 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.194309 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gtnwl"] Jul 14 13:32:23.197860 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.197825 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gtnwl" Jul 14 13:32:23.201294 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:32:23.201260 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod218fb2da_1c3e_4b3b_984a_0861060202a3.slice/crio-362ffb8803438567d19bcb12c63593f7bdf2d79d36bcfc8445baca806ab676e0 WatchSource:0}: Error finding container 362ffb8803438567d19bcb12c63593f7bdf2d79d36bcfc8445baca806ab676e0: Status 404 returned error can't find the container with id 362ffb8803438567d19bcb12c63593f7bdf2d79d36bcfc8445baca806ab676e0 Jul 14 13:32:23.222513 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.222478 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gtnwl"] Jul 14 13:32:23.234048 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.234022 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 14 13:32:23.295576 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.295482 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6kmws"] Jul 14 13:32:23.298145 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.298119 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.310043 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.310022 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 14 13:32:23.310280 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.310270 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-rxxx8\"" Jul 14 13:32:23.310359 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.310280 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 14 13:32:23.310422 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.310409 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 14 13:32:23.310514 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.310492 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 14 13:32:23.318070 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.318044 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6kmws"] Jul 14 13:32:23.333864 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.333831 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvqj4\" (UniqueName: \"kubernetes.io/projected/0b8ca6b7-4548-463a-a764-cbd5f9a5dbdb-kube-api-access-qvqj4\") pod \"model-serving-api-54d89bd5fc-gtnwl\" (UID: \"0b8ca6b7-4548-463a-a764-cbd5f9a5dbdb\") " pod="opendatahub/model-serving-api-54d89bd5fc-gtnwl" Jul 14 13:32:23.334001 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.333871 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b8ca6b7-4548-463a-a764-cbd5f9a5dbdb-tls-certs\") pod \"model-serving-api-54d89bd5fc-gtnwl\" (UID: \"0b8ca6b7-4548-463a-a764-cbd5f9a5dbdb\") " pod="opendatahub/model-serving-api-54d89bd5fc-gtnwl" Jul 14 13:32:23.406593 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.406544 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:23.434629 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.434585 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94594fff-6b06-4132-b97c-ced793cd0339-istio-token\") pod \"istiod-openshift-gateway-7574459656-6kmws\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.434629 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.434631 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/94594fff-6b06-4132-b97c-ced793cd0339-local-certs\") pod \"istiod-openshift-gateway-7574459656-6kmws\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.434829 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.434665 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/94594fff-6b06-4132-b97c-ced793cd0339-cacerts\") pod \"istiod-openshift-gateway-7574459656-6kmws\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.434829 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.434733 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvqj4\" (UniqueName: \"kubernetes.io/projected/0b8ca6b7-4548-463a-a764-cbd5f9a5dbdb-kube-api-access-qvqj4\") pod \"model-serving-api-54d89bd5fc-gtnwl\" (UID: \"0b8ca6b7-4548-463a-a764-cbd5f9a5dbdb\") " pod="opendatahub/model-serving-api-54d89bd5fc-gtnwl" Jul 14 13:32:23.434829 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.434785 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b8ca6b7-4548-463a-a764-cbd5f9a5dbdb-tls-certs\") pod \"model-serving-api-54d89bd5fc-gtnwl\" (UID: \"0b8ca6b7-4548-463a-a764-cbd5f9a5dbdb\") " pod="opendatahub/model-serving-api-54d89bd5fc-gtnwl" Jul 14 13:32:23.434829 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.434811 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l8gw\" (UniqueName: \"kubernetes.io/projected/94594fff-6b06-4132-b97c-ced793cd0339-kube-api-access-2l8gw\") pod \"istiod-openshift-gateway-7574459656-6kmws\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.435004 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.434841 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/94594fff-6b06-4132-b97c-ced793cd0339-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-6kmws\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.435004 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.434873 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/94594fff-6b06-4132-b97c-ced793cd0339-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-6kmws\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.435004 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.434905 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/94594fff-6b06-4132-b97c-ced793cd0339-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-6kmws\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.437389 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.437362 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b8ca6b7-4548-463a-a764-cbd5f9a5dbdb-tls-certs\") pod \"model-serving-api-54d89bd5fc-gtnwl\" (UID: \"0b8ca6b7-4548-463a-a764-cbd5f9a5dbdb\") " pod="opendatahub/model-serving-api-54d89bd5fc-gtnwl" Jul 14 13:32:23.469527 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.469493 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvqj4\" (UniqueName: \"kubernetes.io/projected/0b8ca6b7-4548-463a-a764-cbd5f9a5dbdb-kube-api-access-qvqj4\") pod \"model-serving-api-54d89bd5fc-gtnwl\" (UID: \"0b8ca6b7-4548-463a-a764-cbd5f9a5dbdb\") " pod="opendatahub/model-serving-api-54d89bd5fc-gtnwl" Jul 14 13:32:23.520308 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.520268 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gtnwl" Jul 14 13:32:23.535904 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.535868 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/94594fff-6b06-4132-b97c-ced793cd0339-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-6kmws\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.536060 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.535920 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/94594fff-6b06-4132-b97c-ced793cd0339-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-6kmws\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.536060 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.535969 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94594fff-6b06-4132-b97c-ced793cd0339-istio-token\") pod \"istiod-openshift-gateway-7574459656-6kmws\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.536060 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.536002 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/94594fff-6b06-4132-b97c-ced793cd0339-local-certs\") pod \"istiod-openshift-gateway-7574459656-6kmws\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.536060 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.536046 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/94594fff-6b06-4132-b97c-ced793cd0339-cacerts\") pod \"istiod-openshift-gateway-7574459656-6kmws\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.536299 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.536126 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l8gw\" (UniqueName: \"kubernetes.io/projected/94594fff-6b06-4132-b97c-ced793cd0339-kube-api-access-2l8gw\") pod \"istiod-openshift-gateway-7574459656-6kmws\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.536299 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.536162 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/94594fff-6b06-4132-b97c-ced793cd0339-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-6kmws\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.536928 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.536897 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/94594fff-6b06-4132-b97c-ced793cd0339-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-6kmws\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.538864 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.538838 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/94594fff-6b06-4132-b97c-ced793cd0339-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-6kmws\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.538975 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.538907 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/94594fff-6b06-4132-b97c-ced793cd0339-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-6kmws\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.538975 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.538943 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/94594fff-6b06-4132-b97c-ced793cd0339-local-certs\") pod \"istiod-openshift-gateway-7574459656-6kmws\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.539102 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.539047 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/94594fff-6b06-4132-b97c-ced793cd0339-cacerts\") pod \"istiod-openshift-gateway-7574459656-6kmws\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.549735 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.549643 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l8gw\" (UniqueName: \"kubernetes.io/projected/94594fff-6b06-4132-b97c-ced793cd0339-kube-api-access-2l8gw\") pod \"istiod-openshift-gateway-7574459656-6kmws\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.552826 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.552798 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94594fff-6b06-4132-b97c-ced793cd0339-istio-token\") pod \"istiod-openshift-gateway-7574459656-6kmws\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.607929 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.607891 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:23.701338 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.701306 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gtnwl"] Jul 14 13:32:23.705510 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:32:23.705474 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b8ca6b7_4548_463a_a764_cbd5f9a5dbdb.slice/crio-e0e1c88d3984e404e60a01fbf354deb3a43bec2392001b3d0ec1a117aba4b5f6 WatchSource:0}: Error finding container e0e1c88d3984e404e60a01fbf354deb3a43bec2392001b3d0ec1a117aba4b5f6: Status 404 returned error can't find the container with id e0e1c88d3984e404e60a01fbf354deb3a43bec2392001b3d0ec1a117aba4b5f6 Jul 14 13:32:23.762878 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.762841 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gtnwl" event={"ID":"0b8ca6b7-4548-463a-a764-cbd5f9a5dbdb","Type":"ContainerStarted","Data":"e0e1c88d3984e404e60a01fbf354deb3a43bec2392001b3d0ec1a117aba4b5f6"} Jul 14 13:32:23.763930 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.763904 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-jx5df" event={"ID":"218fb2da-1c3e-4b3b-984a-0861060202a3","Type":"ContainerStarted","Data":"362ffb8803438567d19bcb12c63593f7bdf2d79d36bcfc8445baca806ab676e0"} Jul 14 13:32:23.767632 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:23.767604 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6kmws"] Jul 14 13:32:23.771840 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:32:23.771814 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94594fff_6b06_4132_b97c_ced793cd0339.slice/crio-73f53588507806ee086053856174a56385140cf767294a5f5ba48b94cd75b587 WatchSource:0}: Error finding container 73f53588507806ee086053856174a56385140cf767294a5f5ba48b94cd75b587: Status 404 returned error can't find the container with id 73f53588507806ee086053856174a56385140cf767294a5f5ba48b94cd75b587 Jul 14 13:32:24.778469 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:24.778365 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" event={"ID":"94594fff-6b06-4132-b97c-ced793cd0339","Type":"ContainerStarted","Data":"73f53588507806ee086053856174a56385140cf767294a5f5ba48b94cd75b587"} Jul 14 13:32:25.298641 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:25.298417 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:28.291759 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:28.291658 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:32:28.291759 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:28.291726 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:32:28.801639 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:28.801605 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-jx5df" event={"ID":"218fb2da-1c3e-4b3b-984a-0861060202a3","Type":"ContainerStarted","Data":"2f425add9a651b716d150b94e3b493f492037bb8a22e255e788a323eb25162e8"} Jul 14 13:32:28.801835 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:28.801735 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-jx5df" Jul 14 13:32:28.803281 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:28.803253 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gtnwl" event={"ID":"0b8ca6b7-4548-463a-a764-cbd5f9a5dbdb","Type":"ContainerStarted","Data":"3cfa0918bc7097884e7171a80767b90ac1c719b353c1c7da33a2b88deb9d4307"} Jul 14 13:32:28.803406 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:28.803357 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-gtnwl" Jul 14 13:32:28.804769 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:28.804748 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-594bccb575-nmtvq" event={"ID":"118356c5-d6ff-4662-b421-c2066cedec88","Type":"ContainerStarted","Data":"79748be232818b10c505edd22d54bb9e312a97746c7f8514f54ae464f1f4da4f"} Jul 14 13:32:28.806341 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:28.806314 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" event={"ID":"94594fff-6b06-4132-b97c-ced793cd0339","Type":"ContainerStarted","Data":"c27f72d70200a917936e814c6367e1838df622914f40908c0e0e802663c0bd17"} Jul 14 13:32:28.806447 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:28.806421 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:28.823868 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:28.823816 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-jx5df" podStartSLOduration=1.731201631 podStartE2EDuration="6.823799005s" totalImagesPullingTime="5.092597374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:32:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:32:28.822671347 +0000 UTC m=+312.349821983" watchObservedRunningTime="2026-07-14 13:32:28.823799005 +0000 UTC m=+312.350949640" Jul 14 13:32:28.845906 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:28.845837 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-594bccb575-nmtvq" podStartSLOduration=1.60824218 podStartE2EDuration="7.84581613s" totalImagesPullingTime="6.23757395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:32:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:32:28.845340141 +0000 UTC m=+312.372490778" watchObservedRunningTime="2026-07-14 13:32:28.84581613 +0000 UTC m=+312.372966766" Jul 14 13:32:28.870513 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:28.870447 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" podStartSLOduration=1.3528205770000001 podStartE2EDuration="5.870422935s" totalImagesPullingTime="4.517602358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:32:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:32:28.869664094 +0000 UTC m=+312.396814732" watchObservedRunningTime="2026-07-14 13:32:28.870422935 +0000 UTC m=+312.397573552" Jul 14 13:32:28.889296 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:28.889248 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-gtnwl" podStartSLOduration=1.311603469 podStartE2EDuration="5.88923085s" totalImagesPullingTime="4.577627381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:32:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:32:28.888122726 +0000 UTC m=+312.415273355" watchObservedRunningTime="2026-07-14 13:32:28.88923085 +0000 UTC m=+312.416381486" Jul 14 13:32:29.720045 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:29.720003 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:29.812647 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:29.812611 2583 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-6kmws container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 13:32:29.812825 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:29.812673 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" podUID="94594fff-6b06-4132-b97c-ced793cd0339" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 13:32:29.828858 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:29.828808 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:30.054957 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.054911 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq"] Jul 14 13:32:30.057464 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.057437 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.061516 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.061493 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-dzkn2\"" Jul 14 13:32:30.071500 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.071467 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq"] Jul 14 13:32:30.097280 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.097246 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e94135bb-cabe-49f7-bf34-9220d510b9a9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.097280 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.097278 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e94135bb-cabe-49f7-bf34-9220d510b9a9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.097474 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.097308 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e94135bb-cabe-49f7-bf34-9220d510b9a9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.097474 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.097408 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e94135bb-cabe-49f7-bf34-9220d510b9a9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.097474 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.097463 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e94135bb-cabe-49f7-bf34-9220d510b9a9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.097578 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.097510 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e94135bb-cabe-49f7-bf34-9220d510b9a9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.097578 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.097552 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e94135bb-cabe-49f7-bf34-9220d510b9a9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.097578 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.097573 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b8wq\" (UniqueName: \"kubernetes.io/projected/e94135bb-cabe-49f7-bf34-9220d510b9a9-kube-api-access-5b8wq\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.097666 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.097612 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e94135bb-cabe-49f7-bf34-9220d510b9a9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.107222 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.107182 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:30.198331 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.198290 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e94135bb-cabe-49f7-bf34-9220d510b9a9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.198500 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.198339 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e94135bb-cabe-49f7-bf34-9220d510b9a9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.198500 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.198376 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e94135bb-cabe-49f7-bf34-9220d510b9a9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.198500 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.198406 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e94135bb-cabe-49f7-bf34-9220d510b9a9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.198500 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.198439 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e94135bb-cabe-49f7-bf34-9220d510b9a9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.198500 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.198461 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b8wq\" (UniqueName: \"kubernetes.io/projected/e94135bb-cabe-49f7-bf34-9220d510b9a9-kube-api-access-5b8wq\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.198500 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.198495 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e94135bb-cabe-49f7-bf34-9220d510b9a9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.198805 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.198541 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e94135bb-cabe-49f7-bf34-9220d510b9a9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.198805 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.198568 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e94135bb-cabe-49f7-bf34-9220d510b9a9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.198805 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.198772 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e94135bb-cabe-49f7-bf34-9220d510b9a9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.198958 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.198819 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e94135bb-cabe-49f7-bf34-9220d510b9a9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.199014 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.198950 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e94135bb-cabe-49f7-bf34-9220d510b9a9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.199212 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.199192 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e94135bb-cabe-49f7-bf34-9220d510b9a9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.199414 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.199392 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e94135bb-cabe-49f7-bf34-9220d510b9a9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.200863 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.200843 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e94135bb-cabe-49f7-bf34-9220d510b9a9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.201007 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.200989 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e94135bb-cabe-49f7-bf34-9220d510b9a9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.214892 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.214852 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b8wq\" (UniqueName: \"kubernetes.io/projected/e94135bb-cabe-49f7-bf34-9220d510b9a9-kube-api-access-5b8wq\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.216121 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.216066 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e94135bb-cabe-49f7-bf34-9220d510b9a9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bw8jq\" (UID: \"e94135bb-cabe-49f7-bf34-9220d510b9a9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.241343 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.241301 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:30.367604 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.367507 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:30.388036 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.388003 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:30.546555 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.546526 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq"] Jul 14 13:32:30.548872 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:32:30.548834 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode94135bb_cabe_49f7_bf34_9220d510b9a9.slice/crio-c97d17f9e995087dd1391ce5255702e6ea515c7f41f9ac3162048ae4ba301328 WatchSource:0}: Error finding container c97d17f9e995087dd1391ce5255702e6ea515c7f41f9ac3162048ae4ba301328: Status 404 returned error can't find the container with id c97d17f9e995087dd1391ce5255702e6ea515c7f41f9ac3162048ae4ba301328 Jul 14 13:32:30.585255 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.585218 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:30.696208 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.696114 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:30.815628 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:30.815593 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" event={"ID":"e94135bb-cabe-49f7-bf34-9220d510b9a9","Type":"ContainerStarted","Data":"c97d17f9e995087dd1391ce5255702e6ea515c7f41f9ac3162048ae4ba301328"} Jul 14 13:32:32.810590 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:32.810549 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:32.811312 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:32.811290 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:32.933387 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:32.933353 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qx7mp"] Jul 14 13:32:32.939101 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:32.939051 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qx7mp" Jul 14 13:32:32.942882 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:32.942854 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 14 13:32:32.943579 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:32.943508 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-48fz7"] Jul 14 13:32:32.943698 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:32.943594 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-wxhmd\"" Jul 14 13:32:32.947194 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:32.947173 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-48fz7" Jul 14 13:32:32.961462 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:32.961438 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 14 13:32:32.962189 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:32.961999 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-lrjh7\"" Jul 14 13:32:32.964161 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:32.964137 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qx7mp"] Jul 14 13:32:32.984405 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:32.984366 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-48fz7"] Jul 14 13:32:33.021096 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.021047 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18984890-7e5d-4556-aadc-1cc54a152630-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-48fz7\" (UID: \"18984890-7e5d-4556-aadc-1cc54a152630\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-48fz7" Jul 14 13:32:33.021292 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.021160 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwkfq\" (UniqueName: \"kubernetes.io/projected/18984890-7e5d-4556-aadc-1cc54a152630-kube-api-access-vwkfq\") pod \"llmisvc-controller-manager-777c6bd7d5-48fz7\" (UID: \"18984890-7e5d-4556-aadc-1cc54a152630\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-48fz7" Jul 14 13:32:33.021292 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.021257 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/109d56a5-7738-4b1f-ab5b-410e96a8aafb-cert\") pod \"kserve-controller-manager-d6f4b65b8-qx7mp\" (UID: \"109d56a5-7738-4b1f-ab5b-410e96a8aafb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qx7mp" Jul 14 13:32:33.021404 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.021327 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42p4z\" (UniqueName: \"kubernetes.io/projected/109d56a5-7738-4b1f-ab5b-410e96a8aafb-kube-api-access-42p4z\") pod \"kserve-controller-manager-d6f4b65b8-qx7mp\" (UID: \"109d56a5-7738-4b1f-ab5b-410e96a8aafb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qx7mp" Jul 14 13:32:33.122113 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.122004 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/109d56a5-7738-4b1f-ab5b-410e96a8aafb-cert\") pod \"kserve-controller-manager-d6f4b65b8-qx7mp\" (UID: \"109d56a5-7738-4b1f-ab5b-410e96a8aafb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qx7mp" Jul 14 13:32:33.122113 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.122073 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42p4z\" (UniqueName: \"kubernetes.io/projected/109d56a5-7738-4b1f-ab5b-410e96a8aafb-kube-api-access-42p4z\") pod \"kserve-controller-manager-d6f4b65b8-qx7mp\" (UID: \"109d56a5-7738-4b1f-ab5b-410e96a8aafb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qx7mp" Jul 14 13:32:33.122328 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.122136 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18984890-7e5d-4556-aadc-1cc54a152630-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-48fz7\" (UID: \"18984890-7e5d-4556-aadc-1cc54a152630\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-48fz7" Jul 14 13:32:33.122328 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.122279 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwkfq\" (UniqueName: \"kubernetes.io/projected/18984890-7e5d-4556-aadc-1cc54a152630-kube-api-access-vwkfq\") pod \"llmisvc-controller-manager-777c6bd7d5-48fz7\" (UID: \"18984890-7e5d-4556-aadc-1cc54a152630\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-48fz7" Jul 14 13:32:33.122328 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:32:33.122296 2583 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 14 13:32:33.122470 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:32:33.122372 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18984890-7e5d-4556-aadc-1cc54a152630-cert podName:18984890-7e5d-4556-aadc-1cc54a152630 nodeName:}" failed. No retries permitted until 2026-07-14 13:32:33.622349671 +0000 UTC m=+317.149500306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/18984890-7e5d-4556-aadc-1cc54a152630-cert") pod "llmisvc-controller-manager-777c6bd7d5-48fz7" (UID: "18984890-7e5d-4556-aadc-1cc54a152630") : secret "llmisvc-webhook-server-cert" not found Jul 14 13:32:33.124472 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.124448 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/109d56a5-7738-4b1f-ab5b-410e96a8aafb-cert\") pod \"kserve-controller-manager-d6f4b65b8-qx7mp\" (UID: \"109d56a5-7738-4b1f-ab5b-410e96a8aafb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qx7mp" Jul 14 13:32:33.137008 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.136975 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:33.138413 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.138385 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwkfq\" (UniqueName: \"kubernetes.io/projected/18984890-7e5d-4556-aadc-1cc54a152630-kube-api-access-vwkfq\") pod \"llmisvc-controller-manager-777c6bd7d5-48fz7\" (UID: \"18984890-7e5d-4556-aadc-1cc54a152630\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-48fz7" Jul 14 13:32:33.138807 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.138790 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42p4z\" (UniqueName: \"kubernetes.io/projected/109d56a5-7738-4b1f-ab5b-410e96a8aafb-kube-api-access-42p4z\") pod \"kserve-controller-manager-d6f4b65b8-qx7mp\" (UID: \"109d56a5-7738-4b1f-ab5b-410e96a8aafb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qx7mp" Jul 14 13:32:33.179435 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.179396 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:32:33.179502 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.179478 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:32:33.179536 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.179507 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:32:33.231135 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.231102 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:33.253103 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.253025 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qx7mp" Jul 14 13:32:33.326002 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.325974 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:33.413980 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.413955 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qx7mp"] Jul 14 13:32:33.416676 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:32:33.416651 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod109d56a5_7738_4b1f_ab5b_410e96a8aafb.slice/crio-f33bd23679bfa0764d6b37a843cb93fd6b6037b6da526e8c1764995f114dcad4 WatchSource:0}: Error finding container f33bd23679bfa0764d6b37a843cb93fd6b6037b6da526e8c1764995f114dcad4: Status 404 returned error can't find the container with id f33bd23679bfa0764d6b37a843cb93fd6b6037b6da526e8c1764995f114dcad4 Jul 14 13:32:33.627823 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.627784 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18984890-7e5d-4556-aadc-1cc54a152630-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-48fz7\" (UID: \"18984890-7e5d-4556-aadc-1cc54a152630\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-48fz7" Jul 14 13:32:33.630312 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.630281 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18984890-7e5d-4556-aadc-1cc54a152630-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-48fz7\" (UID: \"18984890-7e5d-4556-aadc-1cc54a152630\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-48fz7" Jul 14 13:32:33.836332 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.836289 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" event={"ID":"e94135bb-cabe-49f7-bf34-9220d510b9a9","Type":"ContainerStarted","Data":"1f690a08992e72f84bac382dac1a4a019c9ad8598f598123fcf6ded56dd8246e"} Jul 14 13:32:33.837479 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.837454 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qx7mp" event={"ID":"109d56a5-7738-4b1f-ab5b-410e96a8aafb","Type":"ContainerStarted","Data":"f33bd23679bfa0764d6b37a843cb93fd6b6037b6da526e8c1764995f114dcad4"} Jul 14 13:32:33.859225 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.859192 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6kmws"] Jul 14 13:32:33.859393 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.859364 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" podUID="94594fff-6b06-4132-b97c-ced793cd0339" containerName="discovery" containerID="cri-o://c27f72d70200a917936e814c6367e1838df622914f40908c0e0e802663c0bd17" gracePeriod=30 Jul 14 13:32:33.864946 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.864923 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-48fz7" Jul 14 13:32:33.886287 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:33.886229 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" podStartSLOduration=1.25835656 podStartE2EDuration="3.886207271s" totalImagesPullingTime="2.627850711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:32:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:32:33.882353004 +0000 UTC m=+317.409503654" watchObservedRunningTime="2026-07-14 13:32:33.886207271 +0000 UTC m=+317.413357908" Jul 14 13:32:34.008593 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.008520 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-48fz7"] Jul 14 13:32:34.011289 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:32:34.011256 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod18984890_7e5d_4556_aadc_1cc54a152630.slice/crio-f0019a1b98cb6ec9d15d78be3893243af737cdf11b12bac22ab62faad354d567 WatchSource:0}: Error finding container f0019a1b98cb6ec9d15d78be3893243af737cdf11b12bac22ab62faad354d567: Status 404 returned error can't find the container with id f0019a1b98cb6ec9d15d78be3893243af737cdf11b12bac22ab62faad354d567 Jul 14 13:32:34.103700 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.103670 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:34.132899 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.132866 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/94594fff-6b06-4132-b97c-ced793cd0339-istio-csr-ca-configmap\") pod \"94594fff-6b06-4132-b97c-ced793cd0339\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " Jul 14 13:32:34.133065 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.133014 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/94594fff-6b06-4132-b97c-ced793cd0339-local-certs\") pod \"94594fff-6b06-4132-b97c-ced793cd0339\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " Jul 14 13:32:34.133240 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.133062 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2l8gw\" (UniqueName: \"kubernetes.io/projected/94594fff-6b06-4132-b97c-ced793cd0339-kube-api-access-2l8gw\") pod \"94594fff-6b06-4132-b97c-ced793cd0339\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " Jul 14 13:32:34.133240 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.133123 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/94594fff-6b06-4132-b97c-ced793cd0339-istio-csr-dns-cert\") pod \"94594fff-6b06-4132-b97c-ced793cd0339\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " Jul 14 13:32:34.133240 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.133189 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/94594fff-6b06-4132-b97c-ced793cd0339-cacerts\") pod \"94594fff-6b06-4132-b97c-ced793cd0339\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " Jul 14 13:32:34.133240 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.133219 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94594fff-6b06-4132-b97c-ced793cd0339-istio-token\") pod \"94594fff-6b06-4132-b97c-ced793cd0339\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " Jul 14 13:32:34.133447 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.133261 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/94594fff-6b06-4132-b97c-ced793cd0339-istio-kubeconfig\") pod \"94594fff-6b06-4132-b97c-ced793cd0339\" (UID: \"94594fff-6b06-4132-b97c-ced793cd0339\") " Jul 14 13:32:34.133447 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.133271 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94594fff-6b06-4132-b97c-ced793cd0339-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "94594fff-6b06-4132-b97c-ced793cd0339" (UID: "94594fff-6b06-4132-b97c-ced793cd0339"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:32:34.133557 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.133521 2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/94594fff-6b06-4132-b97c-ced793cd0339-istio-csr-ca-configmap\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:32:34.137010 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.136968 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94594fff-6b06-4132-b97c-ced793cd0339-local-certs" (OuterVolumeSpecName: "local-certs") pod "94594fff-6b06-4132-b97c-ced793cd0339" (UID: "94594fff-6b06-4132-b97c-ced793cd0339"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:32:34.137548 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.137510 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94594fff-6b06-4132-b97c-ced793cd0339-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "94594fff-6b06-4132-b97c-ced793cd0339" (UID: "94594fff-6b06-4132-b97c-ced793cd0339"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:32:34.137641 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.137539 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94594fff-6b06-4132-b97c-ced793cd0339-istio-token" (OuterVolumeSpecName: "istio-token") pod "94594fff-6b06-4132-b97c-ced793cd0339" (UID: "94594fff-6b06-4132-b97c-ced793cd0339"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:32:34.138341 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.137768 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94594fff-6b06-4132-b97c-ced793cd0339-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "94594fff-6b06-4132-b97c-ced793cd0339" (UID: "94594fff-6b06-4132-b97c-ced793cd0339"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:32:34.138565 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.138534 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94594fff-6b06-4132-b97c-ced793cd0339-cacerts" (OuterVolumeSpecName: "cacerts") pod "94594fff-6b06-4132-b97c-ced793cd0339" (UID: "94594fff-6b06-4132-b97c-ced793cd0339"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:32:34.139011 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.138986 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94594fff-6b06-4132-b97c-ced793cd0339-kube-api-access-2l8gw" (OuterVolumeSpecName: "kube-api-access-2l8gw") pod "94594fff-6b06-4132-b97c-ced793cd0339" (UID: "94594fff-6b06-4132-b97c-ced793cd0339"). InnerVolumeSpecName "kube-api-access-2l8gw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:32:34.234635 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.234598 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2l8gw\" (UniqueName: \"kubernetes.io/projected/94594fff-6b06-4132-b97c-ced793cd0339-kube-api-access-2l8gw\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:32:34.234635 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.234629 2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/94594fff-6b06-4132-b97c-ced793cd0339-istio-csr-dns-cert\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:32:34.234635 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.234638 2583 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/94594fff-6b06-4132-b97c-ced793cd0339-cacerts\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:32:34.234851 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.234647 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94594fff-6b06-4132-b97c-ced793cd0339-istio-token\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:32:34.234851 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.234656 2583 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/94594fff-6b06-4132-b97c-ced793cd0339-istio-kubeconfig\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:32:34.234851 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.234664 2583 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/94594fff-6b06-4132-b97c-ced793cd0339-local-certs\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:32:34.254958 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.254919 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:34.357260 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.357156 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:34.368052 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.367998 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:34.373629 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.373596 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:34.448452 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.448417 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:32:34.842959 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.842928 2583 generic.go:356] "Generic (PLEG): container finished" podID="94594fff-6b06-4132-b97c-ced793cd0339" containerID="c27f72d70200a917936e814c6367e1838df622914f40908c0e0e802663c0bd17" exitCode=0 Jul 14 13:32:34.843385 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.842972 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" Jul 14 13:32:34.843385 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.843019 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" event={"ID":"94594fff-6b06-4132-b97c-ced793cd0339","Type":"ContainerDied","Data":"c27f72d70200a917936e814c6367e1838df622914f40908c0e0e802663c0bd17"} Jul 14 13:32:34.843385 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.843108 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-6kmws" event={"ID":"94594fff-6b06-4132-b97c-ced793cd0339","Type":"ContainerDied","Data":"73f53588507806ee086053856174a56385140cf767294a5f5ba48b94cd75b587"} Jul 14 13:32:34.843385 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.843149 2583 scope.go:117] "RemoveContainer" containerID="c27f72d70200a917936e814c6367e1838df622914f40908c0e0e802663c0bd17" Jul 14 13:32:34.844624 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.844399 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-48fz7" event={"ID":"18984890-7e5d-4556-aadc-1cc54a152630","Type":"ContainerStarted","Data":"f0019a1b98cb6ec9d15d78be3893243af737cdf11b12bac22ab62faad354d567"} Jul 14 13:32:34.844624 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.844604 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:34.846315 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.846292 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bw8jq" Jul 14 13:32:34.859588 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.859571 2583 scope.go:117] "RemoveContainer" containerID="c27f72d70200a917936e814c6367e1838df622914f40908c0e0e802663c0bd17" Jul 14 13:32:34.859925 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:32:34.859904 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c27f72d70200a917936e814c6367e1838df622914f40908c0e0e802663c0bd17\": container with ID starting with c27f72d70200a917936e814c6367e1838df622914f40908c0e0e802663c0bd17 not found: ID does not exist" containerID="c27f72d70200a917936e814c6367e1838df622914f40908c0e0e802663c0bd17" Jul 14 13:32:34.859982 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.859934 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c27f72d70200a917936e814c6367e1838df622914f40908c0e0e802663c0bd17"} err="failed to get container status \"c27f72d70200a917936e814c6367e1838df622914f40908c0e0e802663c0bd17\": rpc error: code = NotFound desc = could not find container \"c27f72d70200a917936e814c6367e1838df622914f40908c0e0e802663c0bd17\": container with ID starting with c27f72d70200a917936e814c6367e1838df622914f40908c0e0e802663c0bd17 not found: ID does not exist" Jul 14 13:32:34.872473 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.872447 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6kmws"] Jul 14 13:32:34.878633 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:34.878608 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-6kmws"] Jul 14 13:32:36.493627 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:36.493537 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94594fff-6b06-4132-b97c-ced793cd0339" path="/var/lib/kubelet/pods/94594fff-6b06-4132-b97c-ced793cd0339/volumes" Jul 14 13:32:36.856867 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:36.856830 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qx7mp" event={"ID":"109d56a5-7738-4b1f-ab5b-410e96a8aafb","Type":"ContainerStarted","Data":"d5bbd980821e0fa7066c8477fd8a80ea72b1fdeb64695390fda2e4bc5c3d20a8"} Jul 14 13:32:36.857156 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:36.857134 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qx7mp" Jul 14 13:32:36.901407 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:36.901356 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qx7mp" podStartSLOduration=2.266485947 podStartE2EDuration="4.901340925s" totalImagesPullingTime="2.634854978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:32:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:32:36.884580249 +0000 UTC m=+320.411730884" watchObservedRunningTime="2026-07-14 13:32:36.901340925 +0000 UTC m=+320.428491560" Jul 14 13:32:37.862702 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:37.862600 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-48fz7" event={"ID":"18984890-7e5d-4556-aadc-1cc54a152630","Type":"ContainerStarted","Data":"fdbdbfd0077f2ce8b91d7f5f4e0e107015a8a1f4e2bb7ffaf761107274cfe1ef"} Jul 14 13:32:37.891762 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:37.891708 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-48fz7" podStartSLOduration=2.300967902 podStartE2EDuration="5.89169143s" totalImagesPullingTime="3.590723528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:32:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:32:37.890688766 +0000 UTC m=+321.417839402" watchObservedRunningTime="2026-07-14 13:32:37.89169143 +0000 UTC m=+321.418842065" Jul 14 13:32:38.866573 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:38.866546 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-48fz7" Jul 14 13:32:39.812514 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:39.812482 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-jx5df" Jul 14 13:32:39.814378 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:32:39.814360 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-gtnwl" Jul 14 13:33:07.868680 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:07.868645 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qx7mp" Jul 14 13:33:09.874758 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:09.874679 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-48fz7" Jul 14 13:33:22.414885 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:22.414841 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:33:22.509163 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:22.509127 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:33:22.573604 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:22.573572 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5rnnc"] Jul 14 13:33:22.573898 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:22.573887 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94594fff-6b06-4132-b97c-ced793cd0339" containerName="discovery" Jul 14 13:33:22.573941 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:22.573899 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="94594fff-6b06-4132-b97c-ced793cd0339" containerName="discovery" Jul 14 13:33:22.573981 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:22.573972 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="94594fff-6b06-4132-b97c-ced793cd0339" containerName="discovery" Jul 14 13:33:22.576890 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:22.576874 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5rnnc" Jul 14 13:33:22.579674 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:22.579653 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 13:33:22.579780 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:22.579727 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 13:33:22.580862 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:22.580846 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-hd7ks\"" Jul 14 13:33:22.587381 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:22.587354 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5rnnc"] Jul 14 13:33:22.633671 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:22.633638 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:33:22.661850 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:22.661816 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b8w5\" (UniqueName: \"kubernetes.io/projected/332b4a22-efb0-4212-82e5-05c84f05f80a-kube-api-access-2b8w5\") pod \"kuadrant-operator-catalog-5rnnc\" (UID: \"332b4a22-efb0-4212-82e5-05c84f05f80a\") " pod="kuadrant-system/kuadrant-operator-catalog-5rnnc" Jul 14 13:33:22.762408 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:22.762370 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b8w5\" (UniqueName: \"kubernetes.io/projected/332b4a22-efb0-4212-82e5-05c84f05f80a-kube-api-access-2b8w5\") pod \"kuadrant-operator-catalog-5rnnc\" (UID: \"332b4a22-efb0-4212-82e5-05c84f05f80a\") " pod="kuadrant-system/kuadrant-operator-catalog-5rnnc" Jul 14 13:33:22.772101 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:22.772055 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b8w5\" (UniqueName: \"kubernetes.io/projected/332b4a22-efb0-4212-82e5-05c84f05f80a-kube-api-access-2b8w5\") pod \"kuadrant-operator-catalog-5rnnc\" (UID: \"332b4a22-efb0-4212-82e5-05c84f05f80a\") " pod="kuadrant-system/kuadrant-operator-catalog-5rnnc" Jul 14 13:33:22.887133 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:22.887062 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5rnnc" Jul 14 13:33:22.934731 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:22.934631 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5rnnc"] Jul 14 13:33:23.024753 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:23.024673 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5rnnc"] Jul 14 13:33:23.028376 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:33:23.028336 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod332b4a22_efb0_4212_82e5_05c84f05f80a.slice/crio-fa33afb879b111ad155f080655453f103baefb7c345305d0a8594505331a882f WatchSource:0}: Error finding container fa33afb879b111ad155f080655453f103baefb7c345305d0a8594505331a882f: Status 404 returned error can't find the container with id fa33afb879b111ad155f080655453f103baefb7c345305d0a8594505331a882f Jul 14 13:33:23.054940 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:23.054905 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5rnnc" event={"ID":"332b4a22-efb0-4212-82e5-05c84f05f80a","Type":"ContainerStarted","Data":"fa33afb879b111ad155f080655453f103baefb7c345305d0a8594505331a882f"} Jul 14 13:33:23.152301 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:23.152266 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-67t8q"] Jul 14 13:33:23.155228 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:23.155210 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-67t8q" Jul 14 13:33:23.165229 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:23.165204 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-67t8q"] Jul 14 13:33:23.266976 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:23.266931 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v676\" (UniqueName: \"kubernetes.io/projected/318072db-4ca4-4cd5-9b92-139d2abc7414-kube-api-access-5v676\") pod \"kuadrant-operator-catalog-67t8q\" (UID: \"318072db-4ca4-4cd5-9b92-139d2abc7414\") " pod="kuadrant-system/kuadrant-operator-catalog-67t8q" Jul 14 13:33:23.367995 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:23.367899 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v676\" (UniqueName: \"kubernetes.io/projected/318072db-4ca4-4cd5-9b92-139d2abc7414-kube-api-access-5v676\") pod \"kuadrant-operator-catalog-67t8q\" (UID: \"318072db-4ca4-4cd5-9b92-139d2abc7414\") " pod="kuadrant-system/kuadrant-operator-catalog-67t8q" Jul 14 13:33:23.386272 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:23.380440 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v676\" (UniqueName: \"kubernetes.io/projected/318072db-4ca4-4cd5-9b92-139d2abc7414-kube-api-access-5v676\") pod \"kuadrant-operator-catalog-67t8q\" (UID: \"318072db-4ca4-4cd5-9b92-139d2abc7414\") " pod="kuadrant-system/kuadrant-operator-catalog-67t8q" Jul 14 13:33:23.464868 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:23.464827 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-67t8q" Jul 14 13:33:23.608206 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:23.608170 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-67t8q"] Jul 14 13:33:23.639681 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:33:23.639600 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod318072db_4ca4_4cd5_9b92_139d2abc7414.slice/crio-425fcec197ef73ceb623b4aad6585a668f81ee3baf1d54cc3126973b704f4c2b WatchSource:0}: Error finding container 425fcec197ef73ceb623b4aad6585a668f81ee3baf1d54cc3126973b704f4c2b: Status 404 returned error can't find the container with id 425fcec197ef73ceb623b4aad6585a668f81ee3baf1d54cc3126973b704f4c2b Jul 14 13:33:24.061979 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:24.061947 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-67t8q" event={"ID":"318072db-4ca4-4cd5-9b92-139d2abc7414","Type":"ContainerStarted","Data":"425fcec197ef73ceb623b4aad6585a668f81ee3baf1d54cc3126973b704f4c2b"} Jul 14 13:33:26.072854 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:26.072819 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5rnnc" event={"ID":"332b4a22-efb0-4212-82e5-05c84f05f80a","Type":"ContainerStarted","Data":"914c23d19e7df05f005d57900b664ea571d4f997d399d76645cf26864a89c555"} Jul 14 13:33:26.073341 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:26.072855 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-5rnnc" podUID="332b4a22-efb0-4212-82e5-05c84f05f80a" containerName="registry-server" containerID="cri-o://914c23d19e7df05f005d57900b664ea571d4f997d399d76645cf26864a89c555" gracePeriod=2 Jul 14 13:33:26.074324 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:26.074296 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-67t8q" event={"ID":"318072db-4ca4-4cd5-9b92-139d2abc7414","Type":"ContainerStarted","Data":"c177a0b35cca5fd39781b4fb8eaacd00d8f5721890e3c1561afe98538c4880cc"} Jul 14 13:33:26.091573 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:26.091525 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5rnnc" podStartSLOduration=1.941657312 podStartE2EDuration="4.091510975s" totalImagesPullingTime="2.149853663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:33:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:33:26.090861025 +0000 UTC m=+369.618011660" watchObservedRunningTime="2026-07-14 13:33:26.091510975 +0000 UTC m=+369.618661611" Jul 14 13:33:26.109650 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:26.109600 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-67t8q" podStartSLOduration=1.56720361 podStartE2EDuration="3.109586022s" totalImagesPullingTime="1.542382412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:33:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:33:26.109002385 +0000 UTC m=+369.636153023" watchObservedRunningTime="2026-07-14 13:33:26.109586022 +0000 UTC m=+369.636736657" Jul 14 13:33:26.318948 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:26.318918 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5rnnc" Jul 14 13:33:26.395552 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:26.395459 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2b8w5\" (UniqueName: \"kubernetes.io/projected/332b4a22-efb0-4212-82e5-05c84f05f80a-kube-api-access-2b8w5\") pod \"332b4a22-efb0-4212-82e5-05c84f05f80a\" (UID: \"332b4a22-efb0-4212-82e5-05c84f05f80a\") " Jul 14 13:33:26.397780 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:26.397748 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/332b4a22-efb0-4212-82e5-05c84f05f80a-kube-api-access-2b8w5" (OuterVolumeSpecName: "kube-api-access-2b8w5") pod "332b4a22-efb0-4212-82e5-05c84f05f80a" (UID: "332b4a22-efb0-4212-82e5-05c84f05f80a"). InnerVolumeSpecName "kube-api-access-2b8w5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:33:26.496517 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:26.496482 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2b8w5\" (UniqueName: \"kubernetes.io/projected/332b4a22-efb0-4212-82e5-05c84f05f80a-kube-api-access-2b8w5\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:33:27.079641 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:27.079605 2583 generic.go:356] "Generic (PLEG): container finished" podID="332b4a22-efb0-4212-82e5-05c84f05f80a" containerID="914c23d19e7df05f005d57900b664ea571d4f997d399d76645cf26864a89c555" exitCode=0 Jul 14 13:33:27.080039 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:27.079656 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5rnnc" Jul 14 13:33:27.080039 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:27.079698 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5rnnc" event={"ID":"332b4a22-efb0-4212-82e5-05c84f05f80a","Type":"ContainerDied","Data":"914c23d19e7df05f005d57900b664ea571d4f997d399d76645cf26864a89c555"} Jul 14 13:33:27.080039 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:27.079747 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5rnnc" event={"ID":"332b4a22-efb0-4212-82e5-05c84f05f80a","Type":"ContainerDied","Data":"fa33afb879b111ad155f080655453f103baefb7c345305d0a8594505331a882f"} Jul 14 13:33:27.080039 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:27.079769 2583 scope.go:117] "RemoveContainer" containerID="914c23d19e7df05f005d57900b664ea571d4f997d399d76645cf26864a89c555" Jul 14 13:33:27.088435 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:27.088415 2583 scope.go:117] "RemoveContainer" containerID="914c23d19e7df05f005d57900b664ea571d4f997d399d76645cf26864a89c555" Jul 14 13:33:27.088729 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:33:27.088707 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"914c23d19e7df05f005d57900b664ea571d4f997d399d76645cf26864a89c555\": container with ID starting with 914c23d19e7df05f005d57900b664ea571d4f997d399d76645cf26864a89c555 not found: ID does not exist" containerID="914c23d19e7df05f005d57900b664ea571d4f997d399d76645cf26864a89c555" Jul 14 13:33:27.088784 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:27.088740 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"914c23d19e7df05f005d57900b664ea571d4f997d399d76645cf26864a89c555"} err="failed to get container status \"914c23d19e7df05f005d57900b664ea571d4f997d399d76645cf26864a89c555\": rpc error: code = NotFound desc = could not find container \"914c23d19e7df05f005d57900b664ea571d4f997d399d76645cf26864a89c555\": container with ID starting with 914c23d19e7df05f005d57900b664ea571d4f997d399d76645cf26864a89c555 not found: ID does not exist" Jul 14 13:33:27.100471 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:27.100442 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5rnnc"] Jul 14 13:33:27.105357 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:27.105329 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5rnnc"] Jul 14 13:33:28.492357 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:28.492322 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="332b4a22-efb0-4212-82e5-05c84f05f80a" path="/var/lib/kubelet/pods/332b4a22-efb0-4212-82e5-05c84f05f80a/volumes" Jul 14 13:33:29.390631 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:29.390602 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:33:33.465005 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:33.464967 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-67t8q" Jul 14 13:33:33.465447 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:33.465052 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-67t8q" Jul 14 13:33:33.487632 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:33.487603 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-67t8q" Jul 14 13:33:34.046953 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:34.046919 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:33:34.132803 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:34.132774 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-67t8q" Jul 14 13:33:50.883870 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:50.883829 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:33:51.504246 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:51.504211 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:33:52.545908 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:52.545872 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:33:52.669483 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:52.669452 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:33:54.738810 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:54.738776 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:33:54.855327 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:54.855297 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:33:55.030992 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:55.030956 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-s7n6g"] Jul 14 13:33:55.031335 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:55.031321 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="332b4a22-efb0-4212-82e5-05c84f05f80a" containerName="registry-server" Jul 14 13:33:55.031384 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:55.031337 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="332b4a22-efb0-4212-82e5-05c84f05f80a" containerName="registry-server" Jul 14 13:33:55.031418 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:55.031402 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="332b4a22-efb0-4212-82e5-05c84f05f80a" containerName="registry-server" Jul 14 13:33:55.035257 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:55.035235 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-s7n6g" Jul 14 13:33:55.038216 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:55.038185 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-rtpbg\"" Jul 14 13:33:55.053854 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:55.053817 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-s7n6g"] Jul 14 13:33:55.144012 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:55.143966 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdsnw\" (UniqueName: \"kubernetes.io/projected/ec2f03ee-0cfb-47d2-92d6-04525e59aa26-kube-api-access-vdsnw\") pod \"authorino-operator-75cc8d5cb7-s7n6g\" (UID: \"ec2f03ee-0cfb-47d2-92d6-04525e59aa26\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-s7n6g" Jul 14 13:33:55.244518 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:55.244479 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdsnw\" (UniqueName: \"kubernetes.io/projected/ec2f03ee-0cfb-47d2-92d6-04525e59aa26-kube-api-access-vdsnw\") pod \"authorino-operator-75cc8d5cb7-s7n6g\" (UID: \"ec2f03ee-0cfb-47d2-92d6-04525e59aa26\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-s7n6g" Jul 14 13:33:55.258904 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:55.258868 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdsnw\" (UniqueName: \"kubernetes.io/projected/ec2f03ee-0cfb-47d2-92d6-04525e59aa26-kube-api-access-vdsnw\") pod \"authorino-operator-75cc8d5cb7-s7n6g\" (UID: \"ec2f03ee-0cfb-47d2-92d6-04525e59aa26\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-s7n6g" Jul 14 13:33:55.345764 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:55.345652 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-s7n6g" Jul 14 13:33:55.491157 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:55.491048 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-s7n6g"] Jul 14 13:33:55.494120 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:33:55.494073 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec2f03ee_0cfb_47d2_92d6_04525e59aa26.slice/crio-8af23698bb2f44c52a6b09810deb032dd151c7c7203bdffba12a6a1a4844f19c WatchSource:0}: Error finding container 8af23698bb2f44c52a6b09810deb032dd151c7c7203bdffba12a6a1a4844f19c: Status 404 returned error can't find the container with id 8af23698bb2f44c52a6b09810deb032dd151c7c7203bdffba12a6a1a4844f19c Jul 14 13:33:56.095844 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:56.095805 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:33:56.205313 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:56.205273 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-s7n6g" event={"ID":"ec2f03ee-0cfb-47d2-92d6-04525e59aa26","Type":"ContainerStarted","Data":"8af23698bb2f44c52a6b09810deb032dd151c7c7203bdffba12a6a1a4844f19c"} Jul 14 13:33:58.010458 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:58.010420 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-2nnxq"] Jul 14 13:33:58.013049 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:58.013030 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2nnxq" Jul 14 13:33:58.015980 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:58.015960 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 14 13:33:58.016047 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:58.016026 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-xgj6n\"" Jul 14 13:33:58.027064 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:58.027039 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-2nnxq"] Jul 14 13:33:58.172519 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:58.172486 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkt8h\" (UniqueName: \"kubernetes.io/projected/cbfcc3c6-4c72-40eb-a830-0b6b01c6947c-kube-api-access-kkt8h\") pod \"dns-operator-controller-manager-864dd78c4c-2nnxq\" (UID: \"cbfcc3c6-4c72-40eb-a830-0b6b01c6947c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2nnxq" Jul 14 13:33:58.215702 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:58.215654 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-s7n6g" event={"ID":"ec2f03ee-0cfb-47d2-92d6-04525e59aa26","Type":"ContainerStarted","Data":"72515d9b20266ccdb62fdf49c12dbe313fa2c3f14fc750e39de8c642a16b40cc"} Jul 14 13:33:58.215869 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:58.215743 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-s7n6g" Jul 14 13:33:58.239024 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:58.238971 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-s7n6g" podStartSLOduration=0.718707421 podStartE2EDuration="3.238954453s" totalImagesPullingTime="2.520247032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:33:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:33:58.238513967 +0000 UTC m=+401.765664613" watchObservedRunningTime="2026-07-14 13:33:58.238954453 +0000 UTC m=+401.766105089" Jul 14 13:33:58.273358 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:58.273263 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkt8h\" (UniqueName: \"kubernetes.io/projected/cbfcc3c6-4c72-40eb-a830-0b6b01c6947c-kube-api-access-kkt8h\") pod \"dns-operator-controller-manager-864dd78c4c-2nnxq\" (UID: \"cbfcc3c6-4c72-40eb-a830-0b6b01c6947c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2nnxq" Jul 14 13:33:58.283745 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:58.283713 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkt8h\" (UniqueName: \"kubernetes.io/projected/cbfcc3c6-4c72-40eb-a830-0b6b01c6947c-kube-api-access-kkt8h\") pod \"dns-operator-controller-manager-864dd78c4c-2nnxq\" (UID: \"cbfcc3c6-4c72-40eb-a830-0b6b01c6947c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2nnxq" Jul 14 13:33:58.324543 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:58.324515 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2nnxq" Jul 14 13:33:58.463940 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:58.463918 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-2nnxq"] Jul 14 13:33:58.466848 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:33:58.466816 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbfcc3c6_4c72_40eb_a830_0b6b01c6947c.slice/crio-2cb1434a0a8977efdb311232e482a64f582536af07c589c6a9fd003016d46373 WatchSource:0}: Error finding container 2cb1434a0a8977efdb311232e482a64f582536af07c589c6a9fd003016d46373: Status 404 returned error can't find the container with id 2cb1434a0a8977efdb311232e482a64f582536af07c589c6a9fd003016d46373 Jul 14 13:33:58.690751 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:58.690627 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:33:58.948115 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:58.948000 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:33:59.221152 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:33:59.221062 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2nnxq" event={"ID":"cbfcc3c6-4c72-40eb-a830-0b6b01c6947c","Type":"ContainerStarted","Data":"2cb1434a0a8977efdb311232e482a64f582536af07c589c6a9fd003016d46373"} Jul 14 13:34:00.794602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:00.794519 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5"] Jul 14 13:34:00.796936 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:00.796913 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5" Jul 14 13:34:00.800038 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:00.800015 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-7v4zq\"" Jul 14 13:34:00.807813 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:00.807793 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5"] Jul 14 13:34:00.893672 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:00.893625 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px44w\" (UniqueName: \"kubernetes.io/projected/42cc63d3-1fee-4cbb-946a-71f8fcf631dc-kube-api-access-px44w\") pod \"limitador-operator-controller-manager-859d7b78bd-8mfp5\" (UID: \"42cc63d3-1fee-4cbb-946a-71f8fcf631dc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5" Jul 14 13:34:00.994998 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:00.994952 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px44w\" (UniqueName: \"kubernetes.io/projected/42cc63d3-1fee-4cbb-946a-71f8fcf631dc-kube-api-access-px44w\") pod \"limitador-operator-controller-manager-859d7b78bd-8mfp5\" (UID: \"42cc63d3-1fee-4cbb-946a-71f8fcf631dc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5" Jul 14 13:34:01.005384 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:01.005352 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px44w\" (UniqueName: \"kubernetes.io/projected/42cc63d3-1fee-4cbb-946a-71f8fcf631dc-kube-api-access-px44w\") pod \"limitador-operator-controller-manager-859d7b78bd-8mfp5\" (UID: \"42cc63d3-1fee-4cbb-946a-71f8fcf631dc\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5" Jul 14 13:34:01.107398 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:01.107299 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5" Jul 14 13:34:01.135238 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:01.135191 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:34:01.240586 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:01.233823 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2nnxq" event={"ID":"cbfcc3c6-4c72-40eb-a830-0b6b01c6947c","Type":"ContainerStarted","Data":"1a7f5a90821b28127af833524af08b94058b4e18012ee544ee7f47eadd2b2bd3"} Jul 14 13:34:01.240586 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:01.234844 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2nnxq" Jul 14 13:34:01.254197 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:01.254128 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5"] Jul 14 13:34:01.256520 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:34:01.256480 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42cc63d3_1fee_4cbb_946a_71f8fcf631dc.slice/crio-19364e9831d05acc771ffce4b011d32d2c589f044e0bd0af164498747bd6b3c0 WatchSource:0}: Error finding container 19364e9831d05acc771ffce4b011d32d2c589f044e0bd0af164498747bd6b3c0: Status 404 returned error can't find the container with id 19364e9831d05acc771ffce4b011d32d2c589f044e0bd0af164498747bd6b3c0 Jul 14 13:34:01.258343 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:01.258289 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2nnxq" podStartSLOduration=2.190531904 podStartE2EDuration="4.258267977s" totalImagesPullingTime="2.067736073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:33:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:34:01.257892029 +0000 UTC m=+404.785042668" watchObservedRunningTime="2026-07-14 13:34:01.258267977 +0000 UTC m=+404.785418616" Jul 14 13:34:01.325232 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:01.325198 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:34:02.241412 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:02.241365 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5" event={"ID":"42cc63d3-1fee-4cbb-946a-71f8fcf631dc","Type":"ContainerStarted","Data":"19364e9831d05acc771ffce4b011d32d2c589f044e0bd0af164498747bd6b3c0"} Jul 14 13:34:03.247992 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:03.247959 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5" event={"ID":"42cc63d3-1fee-4cbb-946a-71f8fcf631dc","Type":"ContainerStarted","Data":"2831a06f45fc1c7b5c50e121f57d7c76dba9b15a9b768b950409058f0ecc0021"} Jul 14 13:34:03.248392 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:03.248110 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5" Jul 14 13:34:03.267983 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:03.267917 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5" podStartSLOduration=1.375784493 podStartE2EDuration="3.267895136s" totalImagesPullingTime="1.892110643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:34:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:34:03.267014922 +0000 UTC m=+406.794165549" watchObservedRunningTime="2026-07-14 13:34:03.267895136 +0000 UTC m=+406.795045776" Jul 14 13:34:03.780901 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:03.780866 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:34:05.294801 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:05.294763 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd"] Jul 14 13:34:05.297670 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:05.297640 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" Jul 14 13:34:05.301878 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:05.301861 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xnwll\"" Jul 14 13:34:05.314140 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:05.314112 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd"] Jul 14 13:34:05.437881 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:05.437846 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frb5w\" (UniqueName: \"kubernetes.io/projected/d40f111c-26be-421c-bca2-d436b47389b7-kube-api-access-frb5w\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w8rfd\" (UID: \"d40f111c-26be-421c-bca2-d436b47389b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" Jul 14 13:34:05.438046 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:05.437921 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d40f111c-26be-421c-bca2-d436b47389b7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w8rfd\" (UID: \"d40f111c-26be-421c-bca2-d436b47389b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" Jul 14 13:34:05.538431 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:05.538394 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frb5w\" (UniqueName: \"kubernetes.io/projected/d40f111c-26be-421c-bca2-d436b47389b7-kube-api-access-frb5w\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w8rfd\" (UID: \"d40f111c-26be-421c-bca2-d436b47389b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" Jul 14 13:34:05.538620 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:05.538448 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d40f111c-26be-421c-bca2-d436b47389b7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w8rfd\" (UID: \"d40f111c-26be-421c-bca2-d436b47389b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" Jul 14 13:34:05.538810 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:05.538792 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d40f111c-26be-421c-bca2-d436b47389b7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w8rfd\" (UID: \"d40f111c-26be-421c-bca2-d436b47389b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" Jul 14 13:34:05.554616 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:05.554544 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frb5w\" (UniqueName: \"kubernetes.io/projected/d40f111c-26be-421c-bca2-d436b47389b7-kube-api-access-frb5w\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w8rfd\" (UID: \"d40f111c-26be-421c-bca2-d436b47389b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" Jul 14 13:34:05.607445 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:05.607402 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" Jul 14 13:34:05.748799 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:05.748759 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd"] Jul 14 13:34:05.751273 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:34:05.751223 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd40f111c_26be_421c_bca2_d436b47389b7.slice/crio-53d9b36718e7e12434766b96c08b02f5d251b075a2555f095670f9b2df864768 WatchSource:0}: Error finding container 53d9b36718e7e12434766b96c08b02f5d251b075a2555f095670f9b2df864768: Status 404 returned error can't find the container with id 53d9b36718e7e12434766b96c08b02f5d251b075a2555f095670f9b2df864768 Jul 14 13:34:06.143691 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:06.143656 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:34:06.242138 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:06.242103 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:34:06.263046 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:06.263007 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" event={"ID":"d40f111c-26be-421c-bca2-d436b47389b7","Type":"ContainerStarted","Data":"53d9b36718e7e12434766b96c08b02f5d251b075a2555f095670f9b2df864768"} Jul 14 13:34:06.344636 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:06.344601 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:34:06.444158 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:06.444052 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:34:06.548147 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:06.548112 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:34:06.650628 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:06.650594 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:34:09.223562 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:09.223519 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-s7n6g" Jul 14 13:34:10.096319 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:10.096285 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:34:10.217864 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:10.217813 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:34:12.296382 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:12.296338 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" event={"ID":"d40f111c-26be-421c-bca2-d436b47389b7","Type":"ContainerStarted","Data":"8a22fc5aa71482dafeb824f518f0b649a49d49249deceeb2fc313c68b6b6749e"} Jul 14 13:34:12.296778 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:12.296480 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" Jul 14 13:34:12.322973 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:12.322920 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" podStartSLOduration=1.5653440440000002 podStartE2EDuration="7.322903802s" totalImagesPullingTime="5.757559758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:34:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:34:12.321527702 +0000 UTC m=+415.848678339" watchObservedRunningTime="2026-07-14 13:34:12.322903802 +0000 UTC m=+415.850054437" Jul 14 13:34:13.250343 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:13.250310 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2nnxq" Jul 14 13:34:14.131338 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:14.131305 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:34:14.232193 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:14.232160 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:34:14.255952 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:14.255921 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5" Jul 14 13:34:14.488956 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:14.488856 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:34:14.720318 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:14.720282 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:34:17.800653 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:17.800613 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:34:17.898196 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:17.898160 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:34:20.767580 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:20.767545 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:34:20.869222 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:20.869184 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:34:23.302797 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:23.302759 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" Jul 14 13:34:25.627534 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.627502 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd"] Jul 14 13:34:25.627902 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.627673 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" podUID="d40f111c-26be-421c-bca2-d436b47389b7" containerName="manager" containerID="cri-o://8a22fc5aa71482dafeb824f518f0b649a49d49249deceeb2fc313c68b6b6749e" gracePeriod=2 Jul 14 13:34:25.637556 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.637525 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd"] Jul 14 13:34:25.639622 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:34:25.639587 2583 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d40f111c-26be-421c-bca2-d436b47389b7\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-w8rfd\": pods \"kuadrant-operator-controller-manager-8684c9ff49-w8rfd\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" Jul 14 13:34:25.656015 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.655988 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5"] Jul 14 13:34:25.656300 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.656264 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5" podUID="42cc63d3-1fee-4cbb-946a-71f8fcf631dc" containerName="manager" containerID="cri-o://2831a06f45fc1c7b5c50e121f57d7c76dba9b15a9b768b950409058f0ecc0021" gracePeriod=2 Jul 14 13:34:25.670890 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.670857 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97"] Jul 14 13:34:25.671417 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.671394 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d40f111c-26be-421c-bca2-d436b47389b7" containerName="manager" Jul 14 13:34:25.671511 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.671420 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d40f111c-26be-421c-bca2-d436b47389b7" containerName="manager" Jul 14 13:34:25.671569 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.671517 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d40f111c-26be-421c-bca2-d436b47389b7" containerName="manager" Jul 14 13:34:25.674404 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.674376 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5"] Jul 14 13:34:25.674528 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.674504 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97" Jul 14 13:34:25.675842 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:34:25.675802 2583 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"42cc63d3-1fee-4cbb-946a-71f8fcf631dc\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-8mfp5\": pods \"limitador-operator-controller-manager-859d7b78bd-8mfp5\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5" Jul 14 13:34:25.681291 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:34:25.681262 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w8rfd\" is forbidden: User \"system:node:ip-10-0-142-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-94.ec2.internal' and this object" podUID="d40f111c-26be-421c-bca2-d436b47389b7" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" Jul 14 13:34:25.686785 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:34:25.686753 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8mfp5\" is forbidden: User \"system:node:ip-10-0-142-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-94.ec2.internal' and this object" podUID="42cc63d3-1fee-4cbb-946a-71f8fcf631dc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5" Jul 14 13:34:25.689321 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:34:25.689295 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w8rfd\" is forbidden: User \"system:node:ip-10-0-142-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-94.ec2.internal' and this object" podUID="d40f111c-26be-421c-bca2-d436b47389b7" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" Jul 14 13:34:25.700035 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.700009 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97"] Jul 14 13:34:25.713806 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.713777 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-78tsv"] Jul 14 13:34:25.714157 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.714143 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42cc63d3-1fee-4cbb-946a-71f8fcf631dc" containerName="manager" Jul 14 13:34:25.714230 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.714157 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="42cc63d3-1fee-4cbb-946a-71f8fcf631dc" containerName="manager" Jul 14 13:34:25.714230 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.714227 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="42cc63d3-1fee-4cbb-946a-71f8fcf631dc" containerName="manager" Jul 14 13:34:25.716094 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.716062 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-78tsv" Jul 14 13:34:25.719553 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.719528 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08d59d28-29fe-44c9-8319-6f451c03ac2f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-44l97\" (UID: \"08d59d28-29fe-44c9-8319-6f451c03ac2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97" Jul 14 13:34:25.719638 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.719593 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndfnm\" (UniqueName: \"kubernetes.io/projected/08d59d28-29fe-44c9-8319-6f451c03ac2f-kube-api-access-ndfnm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-44l97\" (UID: \"08d59d28-29fe-44c9-8319-6f451c03ac2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97" Jul 14 13:34:25.750588 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:34:25.750554 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8mfp5\" is forbidden: User \"system:node:ip-10-0-142-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-94.ec2.internal' and this object" podUID="42cc63d3-1fee-4cbb-946a-71f8fcf631dc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5" Jul 14 13:34:25.763803 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.763776 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-78tsv"] Jul 14 13:34:25.769857 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:34:25.769825 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w8rfd\" is forbidden: User \"system:node:ip-10-0-142-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-94.ec2.internal' and this object" podUID="d40f111c-26be-421c-bca2-d436b47389b7" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" Jul 14 13:34:25.820380 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.820347 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08d59d28-29fe-44c9-8319-6f451c03ac2f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-44l97\" (UID: \"08d59d28-29fe-44c9-8319-6f451c03ac2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97" Jul 14 13:34:25.820583 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.820403 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndfnm\" (UniqueName: \"kubernetes.io/projected/08d59d28-29fe-44c9-8319-6f451c03ac2f-kube-api-access-ndfnm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-44l97\" (UID: \"08d59d28-29fe-44c9-8319-6f451c03ac2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97" Jul 14 13:34:25.820583 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.820433 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phtgh\" (UniqueName: \"kubernetes.io/projected/2fa7beda-73cd-4e50-9a1d-0112b430a57c-kube-api-access-phtgh\") pod \"limitador-operator-controller-manager-859d7b78bd-78tsv\" (UID: \"2fa7beda-73cd-4e50-9a1d-0112b430a57c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-78tsv" Jul 14 13:34:25.820722 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.820700 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08d59d28-29fe-44c9-8319-6f451c03ac2f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-44l97\" (UID: \"08d59d28-29fe-44c9-8319-6f451c03ac2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97" Jul 14 13:34:25.838736 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.838708 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndfnm\" (UniqueName: \"kubernetes.io/projected/08d59d28-29fe-44c9-8319-6f451c03ac2f-kube-api-access-ndfnm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-44l97\" (UID: \"08d59d28-29fe-44c9-8319-6f451c03ac2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97" Jul 14 13:34:25.899349 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.899326 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5" Jul 14 13:34:25.902145 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.902123 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" Jul 14 13:34:25.905161 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:34:25.905137 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8mfp5\" is forbidden: User \"system:node:ip-10-0-142-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-94.ec2.internal' and this object" podUID="42cc63d3-1fee-4cbb-946a-71f8fcf631dc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5" Jul 14 13:34:25.909171 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:34:25.909145 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w8rfd\" is forbidden: User \"system:node:ip-10-0-142-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-94.ec2.internal' and this object" podUID="d40f111c-26be-421c-bca2-d436b47389b7" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" Jul 14 13:34:25.912913 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:34:25.912888 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8mfp5\" is forbidden: User \"system:node:ip-10-0-142-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-94.ec2.internal' and this object" podUID="42cc63d3-1fee-4cbb-946a-71f8fcf631dc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5" Jul 14 13:34:25.915489 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:34:25.915466 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w8rfd\" is forbidden: User \"system:node:ip-10-0-142-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-94.ec2.internal' and this object" podUID="d40f111c-26be-421c-bca2-d436b47389b7" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" Jul 14 13:34:25.921848 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.921827 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phtgh\" (UniqueName: \"kubernetes.io/projected/2fa7beda-73cd-4e50-9a1d-0112b430a57c-kube-api-access-phtgh\") pod \"limitador-operator-controller-manager-859d7b78bd-78tsv\" (UID: \"2fa7beda-73cd-4e50-9a1d-0112b430a57c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-78tsv" Jul 14 13:34:25.940932 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:25.940899 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phtgh\" (UniqueName: \"kubernetes.io/projected/2fa7beda-73cd-4e50-9a1d-0112b430a57c-kube-api-access-phtgh\") pod \"limitador-operator-controller-manager-859d7b78bd-78tsv\" (UID: \"2fa7beda-73cd-4e50-9a1d-0112b430a57c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-78tsv" Jul 14 13:34:26.022534 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.022503 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-px44w\" (UniqueName: \"kubernetes.io/projected/42cc63d3-1fee-4cbb-946a-71f8fcf631dc-kube-api-access-px44w\") pod \"42cc63d3-1fee-4cbb-946a-71f8fcf631dc\" (UID: \"42cc63d3-1fee-4cbb-946a-71f8fcf631dc\") " Jul 14 13:34:26.022534 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.022536 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frb5w\" (UniqueName: \"kubernetes.io/projected/d40f111c-26be-421c-bca2-d436b47389b7-kube-api-access-frb5w\") pod \"d40f111c-26be-421c-bca2-d436b47389b7\" (UID: \"d40f111c-26be-421c-bca2-d436b47389b7\") " Jul 14 13:34:26.022755 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.022561 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d40f111c-26be-421c-bca2-d436b47389b7-extensions-socket-volume\") pod \"d40f111c-26be-421c-bca2-d436b47389b7\" (UID: \"d40f111c-26be-421c-bca2-d436b47389b7\") " Jul 14 13:34:26.023001 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.022972 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d40f111c-26be-421c-bca2-d436b47389b7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d40f111c-26be-421c-bca2-d436b47389b7" (UID: "d40f111c-26be-421c-bca2-d436b47389b7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:34:26.024643 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.024617 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42cc63d3-1fee-4cbb-946a-71f8fcf631dc-kube-api-access-px44w" (OuterVolumeSpecName: "kube-api-access-px44w") pod "42cc63d3-1fee-4cbb-946a-71f8fcf631dc" (UID: "42cc63d3-1fee-4cbb-946a-71f8fcf631dc"). InnerVolumeSpecName "kube-api-access-px44w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:34:26.024707 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.024617 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d40f111c-26be-421c-bca2-d436b47389b7-kube-api-access-frb5w" (OuterVolumeSpecName: "kube-api-access-frb5w") pod "d40f111c-26be-421c-bca2-d436b47389b7" (UID: "d40f111c-26be-421c-bca2-d436b47389b7"). InnerVolumeSpecName "kube-api-access-frb5w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:34:26.056975 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.056950 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97" Jul 14 13:34:26.064588 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.064561 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-78tsv" Jul 14 13:34:26.124348 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.124276 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-px44w\" (UniqueName: \"kubernetes.io/projected/42cc63d3-1fee-4cbb-946a-71f8fcf631dc-kube-api-access-px44w\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:34:26.124348 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.124311 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frb5w\" (UniqueName: \"kubernetes.io/projected/d40f111c-26be-421c-bca2-d436b47389b7-kube-api-access-frb5w\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:34:26.124348 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.124326 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d40f111c-26be-421c-bca2-d436b47389b7-extensions-socket-volume\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:34:26.204461 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.204373 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97"] Jul 14 13:34:26.207357 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:34:26.207323 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08d59d28_29fe_44c9_8319_6f451c03ac2f.slice/crio-720c32ca21c06a30b85250bd99e994c4e0105c09732afd8dbe16e45e3c8b2994 WatchSource:0}: Error finding container 720c32ca21c06a30b85250bd99e994c4e0105c09732afd8dbe16e45e3c8b2994: Status 404 returned error can't find the container with id 720c32ca21c06a30b85250bd99e994c4e0105c09732afd8dbe16e45e3c8b2994 Jul 14 13:34:26.239944 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.239921 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-78tsv"] Jul 14 13:34:26.242648 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:34:26.242619 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fa7beda_73cd_4e50_9a1d_0112b430a57c.slice/crio-5642983a47ba20e5b21dc731ce1acbd84ec024e868f47c76fa6eb321cada304e WatchSource:0}: Error finding container 5642983a47ba20e5b21dc731ce1acbd84ec024e868f47c76fa6eb321cada304e: Status 404 returned error can't find the container with id 5642983a47ba20e5b21dc731ce1acbd84ec024e868f47c76fa6eb321cada304e Jul 14 13:34:26.367037 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.366997 2583 generic.go:356] "Generic (PLEG): container finished" podID="42cc63d3-1fee-4cbb-946a-71f8fcf631dc" containerID="2831a06f45fc1c7b5c50e121f57d7c76dba9b15a9b768b950409058f0ecc0021" exitCode=0 Jul 14 13:34:26.367217 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.367044 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5" Jul 14 13:34:26.367217 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.367126 2583 scope.go:117] "RemoveContainer" containerID="2831a06f45fc1c7b5c50e121f57d7c76dba9b15a9b768b950409058f0ecc0021" Jul 14 13:34:26.368929 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.368903 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-78tsv" event={"ID":"2fa7beda-73cd-4e50-9a1d-0112b430a57c","Type":"ContainerStarted","Data":"a8001e9ffccdaa9015fac8fcd269504aa95e5581cbb2888b0778033f87c3ef5a"} Jul 14 13:34:26.369453 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.369418 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-78tsv" event={"ID":"2fa7beda-73cd-4e50-9a1d-0112b430a57c","Type":"ContainerStarted","Data":"5642983a47ba20e5b21dc731ce1acbd84ec024e868f47c76fa6eb321cada304e"} Jul 14 13:34:26.370034 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:34:26.369993 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w8rfd\" is forbidden: User \"system:node:ip-10-0-142-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-94.ec2.internal' and this object" podUID="d40f111c-26be-421c-bca2-d436b47389b7" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" Jul 14 13:34:26.373184 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:34:26.372331 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8mfp5\" is forbidden: User \"system:node:ip-10-0-142-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-94.ec2.internal' and this object" podUID="42cc63d3-1fee-4cbb-946a-71f8fcf631dc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5" Jul 14 13:34:26.376305 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:34:26.374964 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8mfp5\" is forbidden: User \"system:node:ip-10-0-142-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-94.ec2.internal' and this object" podUID="42cc63d3-1fee-4cbb-946a-71f8fcf631dc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5" Jul 14 13:34:26.378184 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:34:26.378145 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w8rfd\" is forbidden: User \"system:node:ip-10-0-142-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-94.ec2.internal' and this object" podUID="d40f111c-26be-421c-bca2-d436b47389b7" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" Jul 14 13:34:26.379244 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.379226 2583 scope.go:117] "RemoveContainer" containerID="2831a06f45fc1c7b5c50e121f57d7c76dba9b15a9b768b950409058f0ecc0021" Jul 14 13:34:26.379305 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.379282 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97" event={"ID":"08d59d28-29fe-44c9-8319-6f451c03ac2f","Type":"ContainerStarted","Data":"a7bc81df1b5f2d6ede0e8496fb283b1a05405908ce04e48de9d0660c546253d9"} Jul 14 13:34:26.379419 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.379372 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97" event={"ID":"08d59d28-29fe-44c9-8319-6f451c03ac2f","Type":"ContainerStarted","Data":"720c32ca21c06a30b85250bd99e994c4e0105c09732afd8dbe16e45e3c8b2994"} Jul 14 13:34:26.379419 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.379399 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97" Jul 14 13:34:26.379513 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:34:26.379495 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2831a06f45fc1c7b5c50e121f57d7c76dba9b15a9b768b950409058f0ecc0021\": container with ID starting with 2831a06f45fc1c7b5c50e121f57d7c76dba9b15a9b768b950409058f0ecc0021 not found: ID does not exist" containerID="2831a06f45fc1c7b5c50e121f57d7c76dba9b15a9b768b950409058f0ecc0021" Jul 14 13:34:26.379547 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.379521 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2831a06f45fc1c7b5c50e121f57d7c76dba9b15a9b768b950409058f0ecc0021"} err="failed to get container status \"2831a06f45fc1c7b5c50e121f57d7c76dba9b15a9b768b950409058f0ecc0021\": rpc error: code = NotFound desc = could not find container \"2831a06f45fc1c7b5c50e121f57d7c76dba9b15a9b768b950409058f0ecc0021\": container with ID starting with 2831a06f45fc1c7b5c50e121f57d7c76dba9b15a9b768b950409058f0ecc0021 not found: ID does not exist" Jul 14 13:34:26.380629 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.380610 2583 generic.go:356] "Generic (PLEG): container finished" podID="d40f111c-26be-421c-bca2-d436b47389b7" containerID="8a22fc5aa71482dafeb824f518f0b649a49d49249deceeb2fc313c68b6b6749e" exitCode=0 Jul 14 13:34:26.380687 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.380651 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" Jul 14 13:34:26.380743 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.380654 2583 scope.go:117] "RemoveContainer" containerID="8a22fc5aa71482dafeb824f518f0b649a49d49249deceeb2fc313c68b6b6749e" Jul 14 13:34:26.391479 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.391457 2583 scope.go:117] "RemoveContainer" containerID="8a22fc5aa71482dafeb824f518f0b649a49d49249deceeb2fc313c68b6b6749e" Jul 14 13:34:26.391910 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:34:26.391891 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a22fc5aa71482dafeb824f518f0b649a49d49249deceeb2fc313c68b6b6749e\": container with ID starting with 8a22fc5aa71482dafeb824f518f0b649a49d49249deceeb2fc313c68b6b6749e not found: ID does not exist" containerID="8a22fc5aa71482dafeb824f518f0b649a49d49249deceeb2fc313c68b6b6749e" Jul 14 13:34:26.391970 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.391917 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a22fc5aa71482dafeb824f518f0b649a49d49249deceeb2fc313c68b6b6749e"} err="failed to get container status \"8a22fc5aa71482dafeb824f518f0b649a49d49249deceeb2fc313c68b6b6749e\": rpc error: code = NotFound desc = could not find container \"8a22fc5aa71482dafeb824f518f0b649a49d49249deceeb2fc313c68b6b6749e\": container with ID starting with 8a22fc5aa71482dafeb824f518f0b649a49d49249deceeb2fc313c68b6b6749e not found: ID does not exist" Jul 14 13:34:26.403062 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.402981 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-78tsv" podStartSLOduration=1.402967469 podStartE2EDuration="1.402967469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:34:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:34:26.40187428 +0000 UTC m=+429.929024917" watchObservedRunningTime="2026-07-14 13:34:26.402967469 +0000 UTC m=+429.930118174" Jul 14 13:34:26.404496 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:34:26.404474 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8mfp5\" is forbidden: User \"system:node:ip-10-0-142-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-94.ec2.internal' and this object" podUID="42cc63d3-1fee-4cbb-946a-71f8fcf631dc" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8mfp5" Jul 14 13:34:26.406810 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:34:26.406787 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w8rfd\" is forbidden: User \"system:node:ip-10-0-142-94.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-94.ec2.internal' and this object" podUID="d40f111c-26be-421c-bca2-d436b47389b7" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8rfd" Jul 14 13:34:26.434639 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.434590 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97" podStartSLOduration=1.434571004 podStartE2EDuration="1.434571004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:34:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:34:26.43267857 +0000 UTC m=+429.959829207" watchObservedRunningTime="2026-07-14 13:34:26.434571004 +0000 UTC m=+429.961721715" Jul 14 13:34:26.492445 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.492402 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42cc63d3-1fee-4cbb-946a-71f8fcf631dc" path="/var/lib/kubelet/pods/42cc63d3-1fee-4cbb-946a-71f8fcf631dc/volumes" Jul 14 13:34:26.492835 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:26.492814 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d40f111c-26be-421c-bca2-d436b47389b7" path="/var/lib/kubelet/pods/d40f111c-26be-421c-bca2-d436b47389b7/volumes" Jul 14 13:34:27.386944 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:27.386916 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-78tsv" Jul 14 13:34:33.872968 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:33.872937 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:34:37.389539 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:37.389509 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97" Jul 14 13:34:38.393892 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:38.393802 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-78tsv" Jul 14 13:34:42.385704 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:42.385667 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97"] Jul 14 13:34:42.386172 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:42.385860 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97" podUID="08d59d28-29fe-44c9-8319-6f451c03ac2f" containerName="manager" containerID="cri-o://a7bc81df1b5f2d6ede0e8496fb283b1a05405908ce04e48de9d0660c546253d9" gracePeriod=10 Jul 14 13:34:42.615033 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:42.615007 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb"] Jul 14 13:34:42.624254 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:42.624224 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb" Jul 14 13:34:42.629505 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:42.629483 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97" Jul 14 13:34:42.634315 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:42.634289 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb"] Jul 14 13:34:42.771377 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:42.771339 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08d59d28-29fe-44c9-8319-6f451c03ac2f-extensions-socket-volume\") pod \"08d59d28-29fe-44c9-8319-6f451c03ac2f\" (UID: \"08d59d28-29fe-44c9-8319-6f451c03ac2f\") " Jul 14 13:34:42.771577 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:42.771432 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndfnm\" (UniqueName: \"kubernetes.io/projected/08d59d28-29fe-44c9-8319-6f451c03ac2f-kube-api-access-ndfnm\") pod \"08d59d28-29fe-44c9-8319-6f451c03ac2f\" (UID: \"08d59d28-29fe-44c9-8319-6f451c03ac2f\") " Jul 14 13:34:42.771577 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:42.771555 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kvt2\" (UniqueName: \"kubernetes.io/projected/d1a6c552-5d1c-4b02-8ecb-ee5f6168902a-kube-api-access-4kvt2\") pod \"kuadrant-operator-controller-manager-86878c769f-jjsxb\" (UID: \"d1a6c552-5d1c-4b02-8ecb-ee5f6168902a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb" Jul 14 13:34:42.771720 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:42.771692 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1a6c552-5d1c-4b02-8ecb-ee5f6168902a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jjsxb\" (UID: \"d1a6c552-5d1c-4b02-8ecb-ee5f6168902a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb" Jul 14 13:34:42.771766 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:42.771738 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08d59d28-29fe-44c9-8319-6f451c03ac2f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "08d59d28-29fe-44c9-8319-6f451c03ac2f" (UID: "08d59d28-29fe-44c9-8319-6f451c03ac2f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:34:42.771828 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:42.771811 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08d59d28-29fe-44c9-8319-6f451c03ac2f-extensions-socket-volume\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:34:42.773517 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:42.773493 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08d59d28-29fe-44c9-8319-6f451c03ac2f-kube-api-access-ndfnm" (OuterVolumeSpecName: "kube-api-access-ndfnm") pod "08d59d28-29fe-44c9-8319-6f451c03ac2f" (UID: "08d59d28-29fe-44c9-8319-6f451c03ac2f"). InnerVolumeSpecName "kube-api-access-ndfnm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:34:42.873123 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:42.873063 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kvt2\" (UniqueName: \"kubernetes.io/projected/d1a6c552-5d1c-4b02-8ecb-ee5f6168902a-kube-api-access-4kvt2\") pod \"kuadrant-operator-controller-manager-86878c769f-jjsxb\" (UID: \"d1a6c552-5d1c-4b02-8ecb-ee5f6168902a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb" Jul 14 13:34:42.873308 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:42.873159 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1a6c552-5d1c-4b02-8ecb-ee5f6168902a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jjsxb\" (UID: \"d1a6c552-5d1c-4b02-8ecb-ee5f6168902a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb" Jul 14 13:34:42.873308 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:42.873205 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndfnm\" (UniqueName: \"kubernetes.io/projected/08d59d28-29fe-44c9-8319-6f451c03ac2f-kube-api-access-ndfnm\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:34:42.873523 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:42.873504 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1a6c552-5d1c-4b02-8ecb-ee5f6168902a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jjsxb\" (UID: \"d1a6c552-5d1c-4b02-8ecb-ee5f6168902a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb" Jul 14 13:34:42.892490 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:42.892463 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kvt2\" (UniqueName: \"kubernetes.io/projected/d1a6c552-5d1c-4b02-8ecb-ee5f6168902a-kube-api-access-4kvt2\") pod \"kuadrant-operator-controller-manager-86878c769f-jjsxb\" (UID: \"d1a6c552-5d1c-4b02-8ecb-ee5f6168902a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb" Jul 14 13:34:42.937553 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:42.937515 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb" Jul 14 13:34:43.080568 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:43.080537 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb"] Jul 14 13:34:43.083443 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:34:43.083417 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1a6c552_5d1c_4b02_8ecb_ee5f6168902a.slice/crio-5037589f39e7ba1bad1930e951dd99cf26ece3c31123d816c11c76490496470a WatchSource:0}: Error finding container 5037589f39e7ba1bad1930e951dd99cf26ece3c31123d816c11c76490496470a: Status 404 returned error can't find the container with id 5037589f39e7ba1bad1930e951dd99cf26ece3c31123d816c11c76490496470a Jul 14 13:34:43.461753 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:43.461656 2583 generic.go:356] "Generic (PLEG): container finished" podID="08d59d28-29fe-44c9-8319-6f451c03ac2f" containerID="a7bc81df1b5f2d6ede0e8496fb283b1a05405908ce04e48de9d0660c546253d9" exitCode=0 Jul 14 13:34:43.461753 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:43.461699 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97" Jul 14 13:34:43.461753 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:43.461744 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97" event={"ID":"08d59d28-29fe-44c9-8319-6f451c03ac2f","Type":"ContainerDied","Data":"a7bc81df1b5f2d6ede0e8496fb283b1a05405908ce04e48de9d0660c546253d9"} Jul 14 13:34:43.462296 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:43.461795 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97" event={"ID":"08d59d28-29fe-44c9-8319-6f451c03ac2f","Type":"ContainerDied","Data":"720c32ca21c06a30b85250bd99e994c4e0105c09732afd8dbe16e45e3c8b2994"} Jul 14 13:34:43.462296 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:43.461819 2583 scope.go:117] "RemoveContainer" containerID="a7bc81df1b5f2d6ede0e8496fb283b1a05405908ce04e48de9d0660c546253d9" Jul 14 13:34:43.463591 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:43.463570 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb" event={"ID":"d1a6c552-5d1c-4b02-8ecb-ee5f6168902a","Type":"ContainerStarted","Data":"2b27643e614ffad5d4d4d850aa3c4f7b8614a63d3a2e35dd5ccdd8a906690211"} Jul 14 13:34:43.463679 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:43.463614 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb" event={"ID":"d1a6c552-5d1c-4b02-8ecb-ee5f6168902a","Type":"ContainerStarted","Data":"5037589f39e7ba1bad1930e951dd99cf26ece3c31123d816c11c76490496470a"} Jul 14 13:34:43.463729 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:43.463714 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb" Jul 14 13:34:43.471871 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:43.471856 2583 scope.go:117] "RemoveContainer" containerID="a7bc81df1b5f2d6ede0e8496fb283b1a05405908ce04e48de9d0660c546253d9" Jul 14 13:34:43.472164 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:34:43.472145 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7bc81df1b5f2d6ede0e8496fb283b1a05405908ce04e48de9d0660c546253d9\": container with ID starting with a7bc81df1b5f2d6ede0e8496fb283b1a05405908ce04e48de9d0660c546253d9 not found: ID does not exist" containerID="a7bc81df1b5f2d6ede0e8496fb283b1a05405908ce04e48de9d0660c546253d9" Jul 14 13:34:43.472238 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:43.472174 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7bc81df1b5f2d6ede0e8496fb283b1a05405908ce04e48de9d0660c546253d9"} err="failed to get container status \"a7bc81df1b5f2d6ede0e8496fb283b1a05405908ce04e48de9d0660c546253d9\": rpc error: code = NotFound desc = could not find container \"a7bc81df1b5f2d6ede0e8496fb283b1a05405908ce04e48de9d0660c546253d9\": container with ID starting with a7bc81df1b5f2d6ede0e8496fb283b1a05405908ce04e48de9d0660c546253d9 not found: ID does not exist" Jul 14 13:34:43.489884 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:43.489838 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb" podStartSLOduration=1.489819694 podStartE2EDuration="1.489819694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:34:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:34:43.48867689 +0000 UTC m=+447.015827527" watchObservedRunningTime="2026-07-14 13:34:43.489819694 +0000 UTC m=+447.016970330" Jul 14 13:34:43.513912 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:43.513879 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97"] Jul 14 13:34:43.523660 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:43.523627 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-44l97"] Jul 14 13:34:44.492590 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:44.492555 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08d59d28-29fe-44c9-8319-6f451c03ac2f" path="/var/lib/kubelet/pods/08d59d28-29fe-44c9-8319-6f451c03ac2f/volumes" Jul 14 13:34:54.472437 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:54.472405 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb" Jul 14 13:34:59.564439 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.564402 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz"] Jul 14 13:34:59.567533 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.565642 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08d59d28-29fe-44c9-8319-6f451c03ac2f" containerName="manager" Jul 14 13:34:59.567533 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.565676 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="08d59d28-29fe-44c9-8319-6f451c03ac2f" containerName="manager" Jul 14 13:34:59.567533 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.565861 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="08d59d28-29fe-44c9-8319-6f451c03ac2f" containerName="manager" Jul 14 13:34:59.572973 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.572945 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.576324 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.576304 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-vmtwd\"" Jul 14 13:34:59.582829 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.582804 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz"] Jul 14 13:34:59.613732 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.613691 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.613732 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.613734 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.613992 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.613755 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.613992 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.613776 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.613992 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.613889 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.613992 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.613927 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwd8n\" (UniqueName: \"kubernetes.io/projected/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-kube-api-access-wwd8n\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.613992 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.613960 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.614243 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.614008 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.614243 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.614051 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.622788 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.622713 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:34:59.714859 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.714818 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.714859 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.714862 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.715116 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.714882 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.715116 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.714905 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.715116 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.714927 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.715116 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.714978 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.715341 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.715135 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwd8n\" (UniqueName: \"kubernetes.io/projected/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-kube-api-access-wwd8n\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.715341 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.715195 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.715341 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.715261 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.715341 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.715315 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.715526 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.715369 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.715526 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.715474 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.715655 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.715630 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.716172 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.716151 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.717290 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.717266 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.717430 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.717413 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.725487 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.725467 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.725577 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.725533 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwd8n\" (UniqueName: \"kubernetes.io/projected/c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92-kube-api-access-wwd8n\") pod \"maas-default-gateway-openshift-default-5896d4dd49-8wswz\" (UID: \"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:34:59.885234 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:34:59.885139 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:35:00.037502 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:00.037467 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz"] Jul 14 13:35:00.039972 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:35:00.039947 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3bdbf8c_eadb_4aeb_ab2e_a5be43ad3a92.slice/crio-8d273dca07eb4d3accc8f2c4aa0f31974c984eeb59f6b38cc4afc0bc9a987067 WatchSource:0}: Error finding container 8d273dca07eb4d3accc8f2c4aa0f31974c984eeb59f6b38cc4afc0bc9a987067: Status 404 returned error can't find the container with id 8d273dca07eb4d3accc8f2c4aa0f31974c984eeb59f6b38cc4afc0bc9a987067 Jul 14 13:35:00.042608 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:00.042578 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:35:00.042690 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:00.042656 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:35:00.042742 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:00.042690 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:35:00.368093 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:00.368019 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:35:00.547424 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:00.547384 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" event={"ID":"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92","Type":"ContainerStarted","Data":"d6babd6983dd770e640f12e53aec45352f1b986376dced5d8957fab643be6b2d"} Jul 14 13:35:00.547602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:00.547442 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" event={"ID":"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92","Type":"ContainerStarted","Data":"8d273dca07eb4d3accc8f2c4aa0f31974c984eeb59f6b38cc4afc0bc9a987067"} Jul 14 13:35:00.572410 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:00.572362 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" podStartSLOduration=1.5723467530000002 podStartE2EDuration="1.572346753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:34:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:35:00.570971608 +0000 UTC m=+464.098122246" watchObservedRunningTime="2026-07-14 13:35:00.572346753 +0000 UTC m=+464.099497388" Jul 14 13:35:00.886057 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:00.886018 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:35:01.890176 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:01.890135 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:35:02.556711 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:02.556680 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:35:02.557709 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:02.557685 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:35:08.651193 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:08.651156 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:35:13.480354 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:13.480315 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:35:13.581619 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:13.581568 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:35:13.659221 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:13.659188 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:35:14.432228 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:14.432194 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:35:14.535110 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:14.535058 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:35:33.458404 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:33.458367 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:35:33.542223 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:33.542189 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:35:33.910596 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:33.910559 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-62xhp"] Jul 14 13:35:33.914354 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:33.914323 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-62xhp" Jul 14 13:35:33.917616 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:33.917589 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 14 13:35:33.917760 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:33.917625 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-sxfnd\"" Jul 14 13:35:33.918913 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:33.918891 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 13:35:33.919030 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:33.918936 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 14 13:35:33.925123 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:33.925099 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-62xhp"] Jul 14 13:35:33.947349 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:33.947302 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:35:34.026540 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:34.026499 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvhcc\" (UniqueName: \"kubernetes.io/projected/b8741364-7e73-47a4-abd2-4fa8bc1b07cd-kube-api-access-fvhcc\") pod \"postgres-69fb4594cf-62xhp\" (UID: \"b8741364-7e73-47a4-abd2-4fa8bc1b07cd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-62xhp" Jul 14 13:35:34.026720 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:34.026558 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b8741364-7e73-47a4-abd2-4fa8bc1b07cd-data\") pod \"postgres-69fb4594cf-62xhp\" (UID: \"b8741364-7e73-47a4-abd2-4fa8bc1b07cd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-62xhp" Jul 14 13:35:34.127686 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:34.127649 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b8741364-7e73-47a4-abd2-4fa8bc1b07cd-data\") pod \"postgres-69fb4594cf-62xhp\" (UID: \"b8741364-7e73-47a4-abd2-4fa8bc1b07cd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-62xhp" Jul 14 13:35:34.127876 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:34.127847 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvhcc\" (UniqueName: \"kubernetes.io/projected/b8741364-7e73-47a4-abd2-4fa8bc1b07cd-kube-api-access-fvhcc\") pod \"postgres-69fb4594cf-62xhp\" (UID: \"b8741364-7e73-47a4-abd2-4fa8bc1b07cd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-62xhp" Jul 14 13:35:34.128160 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:34.128135 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b8741364-7e73-47a4-abd2-4fa8bc1b07cd-data\") pod \"postgres-69fb4594cf-62xhp\" (UID: \"b8741364-7e73-47a4-abd2-4fa8bc1b07cd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-62xhp" Jul 14 13:35:34.154676 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:34.154639 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvhcc\" (UniqueName: \"kubernetes.io/projected/b8741364-7e73-47a4-abd2-4fa8bc1b07cd-kube-api-access-fvhcc\") pod \"postgres-69fb4594cf-62xhp\" (UID: \"b8741364-7e73-47a4-abd2-4fa8bc1b07cd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-62xhp" Jul 14 13:35:34.225997 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:34.225904 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-62xhp" Jul 14 13:35:34.362256 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:34.362225 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-62xhp"] Jul 14 13:35:34.365009 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:35:34.364974 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8741364_7e73_47a4_abd2_4fa8bc1b07cd.slice/crio-a44e245a59415982cd354d20dfe82018202dfdd3a87666b7ac2c2066296fda5e WatchSource:0}: Error finding container a44e245a59415982cd354d20dfe82018202dfdd3a87666b7ac2c2066296fda5e: Status 404 returned error can't find the container with id a44e245a59415982cd354d20dfe82018202dfdd3a87666b7ac2c2066296fda5e Jul 14 13:35:34.704038 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:34.703996 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-62xhp" event={"ID":"b8741364-7e73-47a4-abd2-4fa8bc1b07cd","Type":"ContainerStarted","Data":"a44e245a59415982cd354d20dfe82018202dfdd3a87666b7ac2c2066296fda5e"} Jul 14 13:35:39.737669 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:39.737632 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-62xhp" event={"ID":"b8741364-7e73-47a4-abd2-4fa8bc1b07cd","Type":"ContainerStarted","Data":"98a0f188e648b21d886bd0fb12f59ba6478fc80e393000d461789448af172d35"} Jul 14 13:35:39.738137 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:39.737724 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-62xhp" Jul 14 13:35:39.758596 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:39.758540 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-62xhp" podStartSLOduration=1.932432001 podStartE2EDuration="6.758522036s" totalImagesPullingTime="4.826090035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:35:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:35:39.758119911 +0000 UTC m=+503.285270542" watchObservedRunningTime="2026-07-14 13:35:39.758522036 +0000 UTC m=+503.285672711" Jul 14 13:35:45.772865 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:45.772833 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-62xhp" Jul 14 13:35:46.144882 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:46.144792 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:35:46.285561 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:46.285517 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:35:53.696279 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:53.696248 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:35:53.854274 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:53.854233 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:35:53.957630 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:53.957535 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-78bfdb59d-86bcv"] Jul 14 13:35:53.974726 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:53.974684 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78bfdb59d-86bcv"] Jul 14 13:35:53.974894 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:53.974792 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78bfdb59d-86bcv" Jul 14 13:35:53.978736 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:53.978706 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 14 13:35:53.979345 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:53.979328 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 14 13:35:53.980024 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:53.980009 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-s5tmc\"" Jul 14 13:35:53.994816 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:53.994790 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:35:54.008250 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:54.008223 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7sjv\" (UniqueName: \"kubernetes.io/projected/75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d-kube-api-access-q7sjv\") pod \"maas-controller-78bfdb59d-86bcv\" (UID: \"75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d\") " pod="opendatahub/maas-controller-78bfdb59d-86bcv" Jul 14 13:35:54.008360 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:54.008253 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d-webhook-cert\") pod \"maas-controller-78bfdb59d-86bcv\" (UID: \"75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d\") " pod="opendatahub/maas-controller-78bfdb59d-86bcv" Jul 14 13:35:54.091031 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:54.090997 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:35:54.109180 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:54.109152 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7sjv\" (UniqueName: \"kubernetes.io/projected/75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d-kube-api-access-q7sjv\") pod \"maas-controller-78bfdb59d-86bcv\" (UID: \"75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d\") " pod="opendatahub/maas-controller-78bfdb59d-86bcv" Jul 14 13:35:54.109310 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:54.109187 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d-webhook-cert\") pod \"maas-controller-78bfdb59d-86bcv\" (UID: \"75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d\") " pod="opendatahub/maas-controller-78bfdb59d-86bcv" Jul 14 13:35:54.111677 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:54.111652 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d-webhook-cert\") pod \"maas-controller-78bfdb59d-86bcv\" (UID: \"75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d\") " pod="opendatahub/maas-controller-78bfdb59d-86bcv" Jul 14 13:35:54.120355 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:54.120324 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7sjv\" (UniqueName: \"kubernetes.io/projected/75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d-kube-api-access-q7sjv\") pod \"maas-controller-78bfdb59d-86bcv\" (UID: \"75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d\") " pod="opendatahub/maas-controller-78bfdb59d-86bcv" Jul 14 13:35:54.288378 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:54.288346 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78bfdb59d-86bcv" Jul 14 13:35:54.426488 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:54.426463 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78bfdb59d-86bcv"] Jul 14 13:35:54.427896 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:35:54.427869 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75d2fa4d_5543_48a8_9ad6_ea8ddd59d63d.slice/crio-23a759cce36d5aa8c6924fd2b31427e530f1662040e26ffe14c201ac1f81330e WatchSource:0}: Error finding container 23a759cce36d5aa8c6924fd2b31427e530f1662040e26ffe14c201ac1f81330e: Status 404 returned error can't find the container with id 23a759cce36d5aa8c6924fd2b31427e530f1662040e26ffe14c201ac1f81330e Jul 14 13:35:54.807458 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:54.807413 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78bfdb59d-86bcv" event={"ID":"75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d","Type":"ContainerStarted","Data":"23a759cce36d5aa8c6924fd2b31427e530f1662040e26ffe14c201ac1f81330e"} Jul 14 13:35:56.908961 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:56.908924 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:35:57.023125 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:57.023075 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:35:57.825315 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:57.825250 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78bfdb59d-86bcv" event={"ID":"75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d","Type":"ContainerStarted","Data":"d925778ba0557d6be736977075aa0dda38a65fdc7f7c88c88983d2f159fa909d"} Jul 14 13:35:57.825522 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:57.825471 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-78bfdb59d-86bcv" Jul 14 13:35:57.848507 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:35:57.848456 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-78bfdb59d-86bcv" podStartSLOduration=2.204418632 podStartE2EDuration="4.848440229s" totalImagesPullingTime="2.644021597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:35:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:35:57.846262535 +0000 UTC m=+521.373413171" watchObservedRunningTime="2026-07-14 13:35:57.848440229 +0000 UTC m=+521.375590863" Jul 14 13:36:08.832762 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:08.832682 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-78bfdb59d-86bcv" Jul 14 13:36:29.728310 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:29.728265 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:36:29.834623 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:29.834589 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:36:29.934993 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:29.934957 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:36:30.028009 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:30.027976 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:36:30.606484 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:30.606439 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:36:30.843454 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:30.843413 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-8nzwn"] Jul 14 13:36:30.894485 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:30.894392 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-8nzwn"] Jul 14 13:36:30.894638 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:30.894527 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-8nzwn" Jul 14 13:36:30.898783 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:30.898762 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 14 13:36:30.898945 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:30.898930 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-vq5p7\"" Jul 14 13:36:30.940287 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:30.940258 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b517ab51-0cb0-47cb-8d09-476de327fe1a-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-8nzwn\" (UID: \"b517ab51-0cb0-47cb-8d09-476de327fe1a\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-8nzwn" Jul 14 13:36:30.940441 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:30.940296 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2krf\" (UniqueName: \"kubernetes.io/projected/b517ab51-0cb0-47cb-8d09-476de327fe1a-kube-api-access-r2krf\") pod \"payload-pre-processing-74584bb9f8-8nzwn\" (UID: \"b517ab51-0cb0-47cb-8d09-476de327fe1a\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-8nzwn" Jul 14 13:36:31.041475 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:31.041435 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b517ab51-0cb0-47cb-8d09-476de327fe1a-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-8nzwn\" (UID: \"b517ab51-0cb0-47cb-8d09-476de327fe1a\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-8nzwn" Jul 14 13:36:31.041629 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:31.041484 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2krf\" (UniqueName: \"kubernetes.io/projected/b517ab51-0cb0-47cb-8d09-476de327fe1a-kube-api-access-r2krf\") pod \"payload-pre-processing-74584bb9f8-8nzwn\" (UID: \"b517ab51-0cb0-47cb-8d09-476de327fe1a\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-8nzwn" Jul 14 13:36:31.042412 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:31.042390 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b517ab51-0cb0-47cb-8d09-476de327fe1a-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-8nzwn\" (UID: \"b517ab51-0cb0-47cb-8d09-476de327fe1a\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-8nzwn" Jul 14 13:36:31.062442 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:31.062404 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2krf\" (UniqueName: \"kubernetes.io/projected/b517ab51-0cb0-47cb-8d09-476de327fe1a-kube-api-access-r2krf\") pod \"payload-pre-processing-74584bb9f8-8nzwn\" (UID: \"b517ab51-0cb0-47cb-8d09-476de327fe1a\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-8nzwn" Jul 14 13:36:31.204588 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:31.204498 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-8nzwn" Jul 14 13:36:31.351497 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:31.351464 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-8nzwn"] Jul 14 13:36:31.354930 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:36:31.354899 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb517ab51_0cb0_47cb_8d09_476de327fe1a.slice/crio-c0efeab81138cf361c74c876d777abf6b512b6bfd71c7d87fdd3779d8025910c WatchSource:0}: Error finding container c0efeab81138cf361c74c876d777abf6b512b6bfd71c7d87fdd3779d8025910c: Status 404 returned error can't find the container with id c0efeab81138cf361c74c876d777abf6b512b6bfd71c7d87fdd3779d8025910c Jul 14 13:36:31.363145 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:31.363116 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:36:31.984057 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:31.984021 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-8nzwn" event={"ID":"b517ab51-0cb0-47cb-8d09-476de327fe1a","Type":"ContainerStarted","Data":"c0efeab81138cf361c74c876d777abf6b512b6bfd71c7d87fdd3779d8025910c"} Jul 14 13:36:32.989551 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:32.989518 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-8nzwn" event={"ID":"b517ab51-0cb0-47cb-8d09-476de327fe1a","Type":"ContainerStarted","Data":"0578e696c8ad4e56b492b9c5ff67ca557df9e6e82c0aa13a6fd2688d47925ed4"} Jul 14 13:36:32.989889 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:32.989646 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-8nzwn" Jul 14 13:36:33.008296 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:33.008239 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-8nzwn" podStartSLOduration=1.466978477 podStartE2EDuration="3.008221099s" totalImagesPullingTime="1.541242622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:36:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:36:33.007148445 +0000 UTC m=+556.534299081" watchObservedRunningTime="2026-07-14 13:36:33.008221099 +0000 UTC m=+556.535371733" Jul 14 13:36:39.443670 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:36:39.443626 2583 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3bdbf8c_eadb_4aeb_ab2e_a5be43ad3a92.slice/crio-conmon-d6babd6983dd770e640f12e53aec45352f1b986376dced5d8957fab643be6b2d.scope\": RecentStats: unable to find data in memory cache]" Jul 14 13:36:40.025067 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:40.025032 2583 generic.go:356] "Generic (PLEG): container finished" podID="c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92" containerID="d6babd6983dd770e640f12e53aec45352f1b986376dced5d8957fab643be6b2d" exitCode=0 Jul 14 13:36:40.025266 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:40.025111 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" event={"ID":"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92","Type":"ContainerDied","Data":"d6babd6983dd770e640f12e53aec45352f1b986376dced5d8957fab643be6b2d"} Jul 14 13:36:40.025689 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:40.025672 2583 scope.go:117] "RemoveContainer" containerID="d6babd6983dd770e640f12e53aec45352f1b986376dced5d8957fab643be6b2d" Jul 14 13:36:40.026864 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:40.026837 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:36:40.026958 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:40.026942 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:36:40.026997 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:40.026982 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:36:40.763072 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:40.763037 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:36:40.851219 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:40.851188 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:36:41.032656 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:41.032570 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" event={"ID":"c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92","Type":"ContainerStarted","Data":"13008a3997a2dc4efda6861ee04a96875c8a85bd8264ef2264e1fadb5a8ad8af"} Jul 14 13:36:41.885976 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:41.885936 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:36:41.886472 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:41.886197 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-8wswz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 14 13:36:41.886472 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:41.886252 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" podUID="c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 14 13:36:42.886321 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:42.886278 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-8wswz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 14 13:36:42.886714 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:42.886341 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" podUID="c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 14 13:36:43.885609 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:43.885572 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-8wswz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 14 13:36:43.885783 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:43.885630 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" podUID="c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 14 13:36:43.995774 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:43.995743 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-8nzwn" Jul 14 13:36:44.885556 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:44.885511 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-8wswz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 14 13:36:44.885769 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:44.885577 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" podUID="c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 14 13:36:45.890522 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:45.890485 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:36:45.890915 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:45.890759 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:36:45.891565 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:45.891545 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-8wswz" Jul 14 13:36:48.768916 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.768877 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8"] Jul 14 13:36:48.771654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.771633 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:48.775573 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.775552 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 14 13:36:48.776033 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.776018 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 14 13:36:48.776347 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.776334 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mmf2v\"" Jul 14 13:36:48.777071 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.777051 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 14 13:36:48.792145 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.792106 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8"] Jul 14 13:36:48.796998 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.796964 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhxpv\" (UniqueName: \"kubernetes.io/projected/ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1-kube-api-access-xhxpv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8\" (UID: \"ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:48.797177 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.797051 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8\" (UID: \"ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:48.797177 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.797150 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8\" (UID: \"ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:48.797286 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.797194 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8\" (UID: \"ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:48.797286 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.797236 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8\" (UID: \"ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:48.797365 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.797305 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8\" (UID: \"ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:48.898220 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.898127 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8\" (UID: \"ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:48.898406 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.898299 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8\" (UID: \"ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:48.898406 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.898354 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhxpv\" (UniqueName: \"kubernetes.io/projected/ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1-kube-api-access-xhxpv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8\" (UID: \"ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:48.898526 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.898441 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8\" (UID: \"ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:48.898526 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.898490 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8\" (UID: \"ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:48.898526 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.898507 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8\" (UID: \"ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:48.898684 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.898545 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8\" (UID: \"ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:48.898743 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.898715 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8\" (UID: \"ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:48.898835 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.898817 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8\" (UID: \"ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:48.900505 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.900480 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8\" (UID: \"ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:48.900805 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.900789 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8\" (UID: \"ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:48.915012 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.914982 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhxpv\" (UniqueName: \"kubernetes.io/projected/ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1-kube-api-access-xhxpv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8\" (UID: \"ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:48.950767 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:48.950727 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:36:49.082138 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:49.082020 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:49.238410 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:49.238374 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8"] Jul 14 13:36:49.242784 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:36:49.242743 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podace2ce02_d8fd_4bf6_a7d2_94bb34db62c1.slice/crio-2853efa5c0a44956b62a44798d2a7b00ffebae689934915d3d1e57c63a098c59 WatchSource:0}: Error finding container 2853efa5c0a44956b62a44798d2a7b00ffebae689934915d3d1e57c63a098c59: Status 404 returned error can't find the container with id 2853efa5c0a44956b62a44798d2a7b00ffebae689934915d3d1e57c63a098c59 Jul 14 13:36:50.089809 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:50.089771 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" event={"ID":"ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1","Type":"ContainerStarted","Data":"2853efa5c0a44956b62a44798d2a7b00ffebae689934915d3d1e57c63a098c59"} Jul 14 13:36:51.095856 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:51.095821 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" event={"ID":"ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1","Type":"ContainerStarted","Data":"122bfdcdbfffa4ae86affe21599c96c65c69c98643bfa0719ecb4f258ee371f1"} Jul 14 13:36:51.118996 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:51.118939 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" podStartSLOduration=1.978904253 podStartE2EDuration="3.118922403s" totalImagesPullingTime="1.14001815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:36:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:36:51.117287902 +0000 UTC m=+574.644438535" watchObservedRunningTime="2026-07-14 13:36:51.118922403 +0000 UTC m=+574.646073039" Jul 14 13:36:55.539550 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:55.539516 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:36:59.083039 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:59.082989 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:59.083571 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:59.083068 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:59.096353 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:59.096318 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:36:59.146174 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:36:59.146135 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8" Jul 14 13:37:14.495880 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.495848 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w"] Jul 14 13:37:14.498340 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.498322 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:14.501358 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.501335 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 14 13:37:14.514497 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.514467 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w"] Jul 14 13:37:14.541409 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.541367 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:37:14.640495 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.640457 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khqq7\" (UniqueName: \"kubernetes.io/projected/037cf229-69eb-4a84-811b-b4eaf0d28fee-kube-api-access-khqq7\") pod \"e2e-trlp-test-simulated-kserve-598466548d-72x8w\" (UID: \"037cf229-69eb-4a84-811b-b4eaf0d28fee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:14.640845 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.640598 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/037cf229-69eb-4a84-811b-b4eaf0d28fee-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-72x8w\" (UID: \"037cf229-69eb-4a84-811b-b4eaf0d28fee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:14.640845 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.640636 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/037cf229-69eb-4a84-811b-b4eaf0d28fee-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-72x8w\" (UID: \"037cf229-69eb-4a84-811b-b4eaf0d28fee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:14.640845 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.640660 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/037cf229-69eb-4a84-811b-b4eaf0d28fee-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-72x8w\" (UID: \"037cf229-69eb-4a84-811b-b4eaf0d28fee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:14.640845 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.640772 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/037cf229-69eb-4a84-811b-b4eaf0d28fee-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-72x8w\" (UID: \"037cf229-69eb-4a84-811b-b4eaf0d28fee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:14.640845 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.640798 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/037cf229-69eb-4a84-811b-b4eaf0d28fee-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-72x8w\" (UID: \"037cf229-69eb-4a84-811b-b4eaf0d28fee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:14.742143 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.742098 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/037cf229-69eb-4a84-811b-b4eaf0d28fee-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-72x8w\" (UID: \"037cf229-69eb-4a84-811b-b4eaf0d28fee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:14.742332 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.742161 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/037cf229-69eb-4a84-811b-b4eaf0d28fee-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-72x8w\" (UID: \"037cf229-69eb-4a84-811b-b4eaf0d28fee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:14.742332 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.742223 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khqq7\" (UniqueName: \"kubernetes.io/projected/037cf229-69eb-4a84-811b-b4eaf0d28fee-kube-api-access-khqq7\") pod \"e2e-trlp-test-simulated-kserve-598466548d-72x8w\" (UID: \"037cf229-69eb-4a84-811b-b4eaf0d28fee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:14.742332 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.742296 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/037cf229-69eb-4a84-811b-b4eaf0d28fee-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-72x8w\" (UID: \"037cf229-69eb-4a84-811b-b4eaf0d28fee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:14.742332 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.742329 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/037cf229-69eb-4a84-811b-b4eaf0d28fee-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-72x8w\" (UID: \"037cf229-69eb-4a84-811b-b4eaf0d28fee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:14.742564 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.742508 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/037cf229-69eb-4a84-811b-b4eaf0d28fee-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-72x8w\" (UID: \"037cf229-69eb-4a84-811b-b4eaf0d28fee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:14.742564 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.742557 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/037cf229-69eb-4a84-811b-b4eaf0d28fee-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-72x8w\" (UID: \"037cf229-69eb-4a84-811b-b4eaf0d28fee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:14.742676 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.742588 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/037cf229-69eb-4a84-811b-b4eaf0d28fee-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-72x8w\" (UID: \"037cf229-69eb-4a84-811b-b4eaf0d28fee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:14.742846 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.742824 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/037cf229-69eb-4a84-811b-b4eaf0d28fee-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-72x8w\" (UID: \"037cf229-69eb-4a84-811b-b4eaf0d28fee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:14.744694 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.744665 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/037cf229-69eb-4a84-811b-b4eaf0d28fee-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-72x8w\" (UID: \"037cf229-69eb-4a84-811b-b4eaf0d28fee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:14.744876 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.744857 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/037cf229-69eb-4a84-811b-b4eaf0d28fee-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-72x8w\" (UID: \"037cf229-69eb-4a84-811b-b4eaf0d28fee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:14.753638 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.753586 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khqq7\" (UniqueName: \"kubernetes.io/projected/037cf229-69eb-4a84-811b-b4eaf0d28fee-kube-api-access-khqq7\") pod \"e2e-trlp-test-simulated-kserve-598466548d-72x8w\" (UID: \"037cf229-69eb-4a84-811b-b4eaf0d28fee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:14.809812 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.809770 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:14.949573 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:14.949547 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w"] Jul 14 13:37:14.951317 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:37:14.951292 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod037cf229_69eb_4a84_811b_b4eaf0d28fee.slice/crio-58e0db542507673bcb3968ce2f55e8c29fc103327059f550a8f2737236bb80da WatchSource:0}: Error finding container 58e0db542507673bcb3968ce2f55e8c29fc103327059f550a8f2737236bb80da: Status 404 returned error can't find the container with id 58e0db542507673bcb3968ce2f55e8c29fc103327059f550a8f2737236bb80da Jul 14 13:37:15.211226 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:15.211196 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" event={"ID":"037cf229-69eb-4a84-811b-b4eaf0d28fee","Type":"ContainerStarted","Data":"58e0db542507673bcb3968ce2f55e8c29fc103327059f550a8f2737236bb80da"} Jul 14 13:37:16.216774 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:16.216732 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" event={"ID":"037cf229-69eb-4a84-811b-b4eaf0d28fee","Type":"ContainerStarted","Data":"21dbec51d8449cb1b1e5392b5de9a58d877af54c522d6e1efac6f502036077ac"} Jul 14 13:37:16.246863 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:16.246795 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" podStartSLOduration=2.018633108 podStartE2EDuration="2.24677543s" totalImagesPullingTime="228.142322ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:37:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:37:16.245445515 +0000 UTC m=+599.772596150" watchObservedRunningTime="2026-07-14 13:37:16.24677543 +0000 UTC m=+599.773926125" Jul 14 13:37:18.253137 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:18.253099 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:37:18.363297 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:18.363267 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 13:37:18.366676 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:18.366653 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 13:37:20.875668 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:20.875626 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:37:24.810924 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:24.810872 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:24.811420 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:24.810960 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:24.823739 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:24.823702 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:25.273478 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:25.273451 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-72x8w" Jul 14 13:37:29.891170 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:29.891135 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:37:31.739066 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:31.739023 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:37:50.951498 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:37:50.951459 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:38:45.352499 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:38:45.352464 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:39:01.736535 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:39:01.736481 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:39:01.928621 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:39:01.928581 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:39:03.004522 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:39:03.004483 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:39:03.104940 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:39:03.104904 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:39:08.943755 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:39:08.943651 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:39:09.072948 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:39:09.072913 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:40:15.914528 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:15.914486 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:40:16.018463 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:16.018421 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-78bfdb59d-86bcv"] Jul 14 13:40:16.018688 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:16.018659 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-78bfdb59d-86bcv" podUID="75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d" containerName="manager" containerID="cri-o://d925778ba0557d6be736977075aa0dda38a65fdc7f7c88c88983d2f159fa909d" gracePeriod=30 Jul 14 13:40:16.268507 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:16.268482 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78bfdb59d-86bcv" Jul 14 13:40:16.323797 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:16.323760 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d-webhook-cert\") pod \"75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d\" (UID: \"75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d\") " Jul 14 13:40:16.326185 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:16.326151 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d" (UID: "75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:40:16.424404 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:16.424365 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7sjv\" (UniqueName: \"kubernetes.io/projected/75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d-kube-api-access-q7sjv\") pod \"75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d\" (UID: \"75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d\") " Jul 14 13:40:16.424595 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:16.424563 2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d-webhook-cert\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:40:16.426406 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:16.426377 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d-kube-api-access-q7sjv" (OuterVolumeSpecName: "kube-api-access-q7sjv") pod "75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d" (UID: "75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d"). InnerVolumeSpecName "kube-api-access-q7sjv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:40:16.525651 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:16.525612 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7sjv\" (UniqueName: \"kubernetes.io/projected/75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d-kube-api-access-q7sjv\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:40:17.090476 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:17.090441 2583 generic.go:356] "Generic (PLEG): container finished" podID="75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d" containerID="d925778ba0557d6be736977075aa0dda38a65fdc7f7c88c88983d2f159fa909d" exitCode=0 Jul 14 13:40:17.090902 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:17.090490 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78bfdb59d-86bcv" Jul 14 13:40:17.090902 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:17.090502 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78bfdb59d-86bcv" event={"ID":"75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d","Type":"ContainerDied","Data":"d925778ba0557d6be736977075aa0dda38a65fdc7f7c88c88983d2f159fa909d"} Jul 14 13:40:17.090902 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:17.090547 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78bfdb59d-86bcv" event={"ID":"75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d","Type":"ContainerDied","Data":"23a759cce36d5aa8c6924fd2b31427e530f1662040e26ffe14c201ac1f81330e"} Jul 14 13:40:17.090902 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:17.090572 2583 scope.go:117] "RemoveContainer" containerID="d925778ba0557d6be736977075aa0dda38a65fdc7f7c88c88983d2f159fa909d" Jul 14 13:40:17.099290 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:17.099271 2583 scope.go:117] "RemoveContainer" containerID="d925778ba0557d6be736977075aa0dda38a65fdc7f7c88c88983d2f159fa909d" Jul 14 13:40:17.099558 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:40:17.099539 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d925778ba0557d6be736977075aa0dda38a65fdc7f7c88c88983d2f159fa909d\": container with ID starting with d925778ba0557d6be736977075aa0dda38a65fdc7f7c88c88983d2f159fa909d not found: ID does not exist" containerID="d925778ba0557d6be736977075aa0dda38a65fdc7f7c88c88983d2f159fa909d" Jul 14 13:40:17.099605 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:17.099569 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d925778ba0557d6be736977075aa0dda38a65fdc7f7c88c88983d2f159fa909d"} err="failed to get container status \"d925778ba0557d6be736977075aa0dda38a65fdc7f7c88c88983d2f159fa909d\": rpc error: code = NotFound desc = could not find container \"d925778ba0557d6be736977075aa0dda38a65fdc7f7c88c88983d2f159fa909d\": container with ID starting with d925778ba0557d6be736977075aa0dda38a65fdc7f7c88c88983d2f159fa909d not found: ID does not exist" Jul 14 13:40:17.116632 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:17.116581 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-78bfdb59d-86bcv"] Jul 14 13:40:17.119723 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:17.119699 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-78bfdb59d-86bcv"] Jul 14 13:40:17.840012 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:17.839975 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:40:17.981714 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:17.981680 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-78bfdb59d-gtpx2"] Jul 14 13:40:17.982113 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:17.982099 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d" containerName="manager" Jul 14 13:40:17.982177 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:17.982115 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d" containerName="manager" Jul 14 13:40:17.982213 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:17.982191 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d" containerName="manager" Jul 14 13:40:17.984878 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:17.984859 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78bfdb59d-gtpx2" Jul 14 13:40:17.989269 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:17.989249 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-s5tmc\"" Jul 14 13:40:17.989401 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:17.989270 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 14 13:40:17.989401 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:17.989254 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 14 13:40:18.000802 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:18.000774 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78bfdb59d-gtpx2"] Jul 14 13:40:18.034726 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:18.034688 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-262tj\" (UniqueName: \"kubernetes.io/projected/6aadcd44-2422-4e08-9bb0-cee729bd6011-kube-api-access-262tj\") pod \"maas-controller-78bfdb59d-gtpx2\" (UID: \"6aadcd44-2422-4e08-9bb0-cee729bd6011\") " pod="opendatahub/maas-controller-78bfdb59d-gtpx2" Jul 14 13:40:18.034726 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:18.034729 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6aadcd44-2422-4e08-9bb0-cee729bd6011-webhook-cert\") pod \"maas-controller-78bfdb59d-gtpx2\" (UID: \"6aadcd44-2422-4e08-9bb0-cee729bd6011\") " pod="opendatahub/maas-controller-78bfdb59d-gtpx2" Jul 14 13:40:18.135476 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:18.135386 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-262tj\" (UniqueName: \"kubernetes.io/projected/6aadcd44-2422-4e08-9bb0-cee729bd6011-kube-api-access-262tj\") pod \"maas-controller-78bfdb59d-gtpx2\" (UID: \"6aadcd44-2422-4e08-9bb0-cee729bd6011\") " pod="opendatahub/maas-controller-78bfdb59d-gtpx2" Jul 14 13:40:18.135476 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:18.135426 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6aadcd44-2422-4e08-9bb0-cee729bd6011-webhook-cert\") pod \"maas-controller-78bfdb59d-gtpx2\" (UID: \"6aadcd44-2422-4e08-9bb0-cee729bd6011\") " pod="opendatahub/maas-controller-78bfdb59d-gtpx2" Jul 14 13:40:18.137941 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:18.137920 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6aadcd44-2422-4e08-9bb0-cee729bd6011-webhook-cert\") pod \"maas-controller-78bfdb59d-gtpx2\" (UID: \"6aadcd44-2422-4e08-9bb0-cee729bd6011\") " pod="opendatahub/maas-controller-78bfdb59d-gtpx2" Jul 14 13:40:18.146645 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:18.146612 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-262tj\" (UniqueName: \"kubernetes.io/projected/6aadcd44-2422-4e08-9bb0-cee729bd6011-kube-api-access-262tj\") pod \"maas-controller-78bfdb59d-gtpx2\" (UID: \"6aadcd44-2422-4e08-9bb0-cee729bd6011\") " pod="opendatahub/maas-controller-78bfdb59d-gtpx2" Jul 14 13:40:18.298710 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:18.298681 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-s5tmc\"" Jul 14 13:40:18.306011 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:18.305993 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78bfdb59d-gtpx2" Jul 14 13:40:18.449209 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:18.449187 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78bfdb59d-gtpx2"] Jul 14 13:40:18.451049 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:40:18.451015 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aadcd44_2422_4e08_9bb0_cee729bd6011.slice/crio-b474a6a90f41e787daa2b4b04469fc1e62c79b61bf3baa269ea333196b7a3cfb WatchSource:0}: Error finding container b474a6a90f41e787daa2b4b04469fc1e62c79b61bf3baa269ea333196b7a3cfb: Status 404 returned error can't find the container with id b474a6a90f41e787daa2b4b04469fc1e62c79b61bf3baa269ea333196b7a3cfb Jul 14 13:40:18.452347 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:18.452330 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 13:40:18.494202 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:18.494173 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d" path="/var/lib/kubelet/pods/75d2fa4d-5543-48a8-9ad6-ea8ddd59d63d/volumes" Jul 14 13:40:18.589983 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:18.589948 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:40:18.919236 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:18.919213 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 14 13:40:19.103131 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:19.103073 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78bfdb59d-gtpx2" event={"ID":"6aadcd44-2422-4e08-9bb0-cee729bd6011","Type":"ContainerStarted","Data":"cb11a3e6d202337cc117dde1dab85bd5a0fc79d959684a49dd63e2b7b93a7cea"} Jul 14 13:40:19.103295 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:19.103157 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78bfdb59d-gtpx2" event={"ID":"6aadcd44-2422-4e08-9bb0-cee729bd6011","Type":"ContainerStarted","Data":"b474a6a90f41e787daa2b4b04469fc1e62c79b61bf3baa269ea333196b7a3cfb"} Jul 14 13:40:19.103295 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:19.103216 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-78bfdb59d-gtpx2" Jul 14 13:40:19.129447 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:19.129396 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-78bfdb59d-gtpx2" podStartSLOduration=1.665804842 podStartE2EDuration="2.129381167s" totalImagesPullingTime="463.576325ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:40:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:40:19.129053213 +0000 UTC m=+782.656203851" watchObservedRunningTime="2026-07-14 13:40:19.129381167 +0000 UTC m=+782.656531802" Jul 14 13:40:30.110073 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:30.110037 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-78bfdb59d-gtpx2" Jul 14 13:40:52.068654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:52.068616 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:40:52.170226 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:52.170188 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:40:58.887423 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:58.887385 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:40:58.981950 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:58.981911 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:40:59.122178 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:40:59.122148 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:41:06.276672 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:41:06.276635 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:41:06.379641 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:41:06.379599 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:41:06.483176 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:41:06.483144 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:41:06.694746 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:41:06.694656 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:41:07.751687 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:41:07.751654 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:41:07.847433 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:41:07.847398 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:41:13.600897 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:41:13.600848 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:41:13.701993 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:41:13.701956 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:42:18.397323 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:42:18.397228 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 13:42:18.399099 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:42:18.397826 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 13:43:58.066865 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:43:58.066829 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:44:09.923212 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:44:09.923173 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:44:11.213329 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:44:11.213293 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:45:25.250432 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:45:25.250351 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb"] Jul 14 13:45:25.250996 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:45:25.250560 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb" podUID="d1a6c552-5d1c-4b02-8ecb-ee5f6168902a" containerName="manager" containerID="cri-o://2b27643e614ffad5d4d4d850aa3c4f7b8614a63d3a2e35dd5ccdd8a906690211" gracePeriod=10 Jul 14 13:45:27.605595 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:45:27.605560 2583 generic.go:356] "Generic (PLEG): container finished" podID="d1a6c552-5d1c-4b02-8ecb-ee5f6168902a" containerID="2b27643e614ffad5d4d4d850aa3c4f7b8614a63d3a2e35dd5ccdd8a906690211" exitCode=0 Jul 14 13:45:27.606022 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:45:27.605594 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb" event={"ID":"d1a6c552-5d1c-4b02-8ecb-ee5f6168902a","Type":"ContainerDied","Data":"2b27643e614ffad5d4d4d850aa3c4f7b8614a63d3a2e35dd5ccdd8a906690211"} Jul 14 13:45:27.712876 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:45:27.712850 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb" Jul 14 13:45:27.883847 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:45:27.883757 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kvt2\" (UniqueName: \"kubernetes.io/projected/d1a6c552-5d1c-4b02-8ecb-ee5f6168902a-kube-api-access-4kvt2\") pod \"d1a6c552-5d1c-4b02-8ecb-ee5f6168902a\" (UID: \"d1a6c552-5d1c-4b02-8ecb-ee5f6168902a\") " Jul 14 13:45:27.883847 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:45:27.883827 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1a6c552-5d1c-4b02-8ecb-ee5f6168902a-extensions-socket-volume\") pod \"d1a6c552-5d1c-4b02-8ecb-ee5f6168902a\" (UID: \"d1a6c552-5d1c-4b02-8ecb-ee5f6168902a\") " Jul 14 13:45:27.884215 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:45:27.884187 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1a6c552-5d1c-4b02-8ecb-ee5f6168902a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d1a6c552-5d1c-4b02-8ecb-ee5f6168902a" (UID: "d1a6c552-5d1c-4b02-8ecb-ee5f6168902a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:45:27.885929 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:45:27.885900 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1a6c552-5d1c-4b02-8ecb-ee5f6168902a-kube-api-access-4kvt2" (OuterVolumeSpecName: "kube-api-access-4kvt2") pod "d1a6c552-5d1c-4b02-8ecb-ee5f6168902a" (UID: "d1a6c552-5d1c-4b02-8ecb-ee5f6168902a"). InnerVolumeSpecName "kube-api-access-4kvt2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:45:27.985489 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:45:27.985447 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1a6c552-5d1c-4b02-8ecb-ee5f6168902a-extensions-socket-volume\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:45:27.985489 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:45:27.985483 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kvt2\" (UniqueName: \"kubernetes.io/projected/d1a6c552-5d1c-4b02-8ecb-ee5f6168902a-kube-api-access-4kvt2\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:45:28.611294 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:45:28.611257 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb" event={"ID":"d1a6c552-5d1c-4b02-8ecb-ee5f6168902a","Type":"ContainerDied","Data":"5037589f39e7ba1bad1930e951dd99cf26ece3c31123d816c11c76490496470a"} Jul 14 13:45:28.611294 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:45:28.611287 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb" Jul 14 13:45:28.611836 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:45:28.611321 2583 scope.go:117] "RemoveContainer" containerID="2b27643e614ffad5d4d4d850aa3c4f7b8614a63d3a2e35dd5ccdd8a906690211" Jul 14 13:45:28.636286 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:45:28.636258 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb"] Jul 14 13:45:28.641866 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:45:28.641843 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jjsxb"] Jul 14 13:45:30.493158 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:45:30.493127 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1a6c552-5d1c-4b02-8ecb-ee5f6168902a" path="/var/lib/kubelet/pods/d1a6c552-5d1c-4b02-8ecb-ee5f6168902a/volumes" Jul 14 13:46:26.680431 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:46:26.680390 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqbhm"] Jul 14 13:46:26.681000 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:46:26.680981 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1a6c552-5d1c-4b02-8ecb-ee5f6168902a" containerName="manager" Jul 14 13:46:26.681102 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:46:26.681003 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1a6c552-5d1c-4b02-8ecb-ee5f6168902a" containerName="manager" Jul 14 13:46:26.681172 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:46:26.681111 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1a6c552-5d1c-4b02-8ecb-ee5f6168902a" containerName="manager" Jul 14 13:46:26.685576 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:46:26.685552 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqbhm" Jul 14 13:46:26.688982 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:46:26.688964 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xnwll\"" Jul 14 13:46:26.697371 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:46:26.697348 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqbhm"] Jul 14 13:46:26.794271 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:46:26.794227 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f82d\" (UniqueName: \"kubernetes.io/projected/6dea9fa8-c924-457d-bc4e-7477e5e8ff9f-kube-api-access-4f82d\") pod \"kuadrant-operator-controller-manager-86878c769f-pqbhm\" (UID: \"6dea9fa8-c924-457d-bc4e-7477e5e8ff9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqbhm" Jul 14 13:46:26.794461 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:46:26.794349 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6dea9fa8-c924-457d-bc4e-7477e5e8ff9f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pqbhm\" (UID: \"6dea9fa8-c924-457d-bc4e-7477e5e8ff9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqbhm" Jul 14 13:46:26.895776 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:46:26.895740 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f82d\" (UniqueName: \"kubernetes.io/projected/6dea9fa8-c924-457d-bc4e-7477e5e8ff9f-kube-api-access-4f82d\") pod \"kuadrant-operator-controller-manager-86878c769f-pqbhm\" (UID: \"6dea9fa8-c924-457d-bc4e-7477e5e8ff9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqbhm" Jul 14 13:46:26.895965 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:46:26.895831 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6dea9fa8-c924-457d-bc4e-7477e5e8ff9f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pqbhm\" (UID: \"6dea9fa8-c924-457d-bc4e-7477e5e8ff9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqbhm" Jul 14 13:46:26.896239 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:46:26.896219 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6dea9fa8-c924-457d-bc4e-7477e5e8ff9f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pqbhm\" (UID: \"6dea9fa8-c924-457d-bc4e-7477e5e8ff9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqbhm" Jul 14 13:46:26.917370 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:46:26.917346 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f82d\" (UniqueName: \"kubernetes.io/projected/6dea9fa8-c924-457d-bc4e-7477e5e8ff9f-kube-api-access-4f82d\") pod \"kuadrant-operator-controller-manager-86878c769f-pqbhm\" (UID: \"6dea9fa8-c924-457d-bc4e-7477e5e8ff9f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqbhm" Jul 14 13:46:26.996885 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:46:26.996858 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqbhm" Jul 14 13:46:27.138444 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:46:27.138416 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqbhm"] Jul 14 13:46:27.140412 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:46:27.140386 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dea9fa8_c924_457d_bc4e_7477e5e8ff9f.slice/crio-019868d89f03e5c31083f4958152b90e3b6f7f42de3c7d80b11789628cc66b05 WatchSource:0}: Error finding container 019868d89f03e5c31083f4958152b90e3b6f7f42de3c7d80b11789628cc66b05: Status 404 returned error can't find the container with id 019868d89f03e5c31083f4958152b90e3b6f7f42de3c7d80b11789628cc66b05 Jul 14 13:46:27.897181 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:46:27.897141 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqbhm" event={"ID":"6dea9fa8-c924-457d-bc4e-7477e5e8ff9f","Type":"ContainerStarted","Data":"6854975addc910f3a2492f7fa44a3c830bb4610c1ab5a66447630a83c478ede7"} Jul 14 13:46:27.897613 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:46:27.897194 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqbhm" event={"ID":"6dea9fa8-c924-457d-bc4e-7477e5e8ff9f","Type":"ContainerStarted","Data":"019868d89f03e5c31083f4958152b90e3b6f7f42de3c7d80b11789628cc66b05"} Jul 14 13:46:27.897613 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:46:27.897253 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqbhm" Jul 14 13:46:27.925956 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:46:27.925904 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqbhm" podStartSLOduration=1.9258863640000001 podStartE2EDuration="1.925886364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:46:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:46:27.9226716 +0000 UTC m=+1151.449822232" watchObservedRunningTime="2026-07-14 13:46:27.925886364 +0000 UTC m=+1151.453037049" Jul 14 13:46:38.903534 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:46:38.903448 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pqbhm" Jul 14 13:47:18.428352 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:47:18.428231 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 13:47:18.432119 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:47:18.428754 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 13:47:30.474860 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:47:30.474822 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:47:30.595435 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:47:30.595403 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:48:48.619371 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:48:48.619332 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:49:12.223863 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:49:12.223824 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:52:03.091949 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:03.091914 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:52:18.459688 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:18.459574 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 13:52:18.463179 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:18.461777 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 13:52:32.425502 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:32.425465 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:52:32.523362 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:32.523326 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:52:32.914877 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:32.914836 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:52:39.776876 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:39.776790 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:52:39.844860 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:39.844821 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:52:39.970297 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:39.970266 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg"] Jul 14 13:52:39.975704 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:39.975672 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:39.979255 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:39.979221 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b0840d76-openshift-default-dockercfg-nfpjs\"" Jul 14 13:52:39.996692 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:39.996661 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg"] Jul 14 13:52:40.004322 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.004293 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:52:40.091644 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.091543 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-workload-certs\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.091644 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.091582 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vpql\" (UniqueName: \"kubernetes.io/projected/116dd55c-1850-4767-8d20-3760e20ea330-kube-api-access-8vpql\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.091644 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.091606 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/116dd55c-1850-4767-8d20-3760e20ea330-istio-token\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.091644 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.091633 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/116dd55c-1850-4767-8d20-3760e20ea330-istiod-ca-cert\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.091965 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.091750 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/116dd55c-1850-4767-8d20-3760e20ea330-istio-podinfo\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.091965 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.091783 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-istio-data\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.091965 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.091871 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-workload-socket\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.091965 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.091911 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-credential-socket\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.091965 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.091951 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-istio-envoy\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.104789 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.104747 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:52:40.192553 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.192512 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-workload-certs\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.192553 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.192552 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vpql\" (UniqueName: \"kubernetes.io/projected/116dd55c-1850-4767-8d20-3760e20ea330-kube-api-access-8vpql\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.192826 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.192570 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/116dd55c-1850-4767-8d20-3760e20ea330-istio-token\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.192826 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.192675 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/116dd55c-1850-4767-8d20-3760e20ea330-istiod-ca-cert\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.192826 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.192723 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/116dd55c-1850-4767-8d20-3760e20ea330-istio-podinfo\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.192826 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.192743 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-istio-data\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.192826 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.192777 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-workload-socket\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.192826 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.192818 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-credential-socket\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.193219 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.192841 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-istio-envoy\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.193219 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.193062 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-workload-certs\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.193219 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.193177 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-istio-data\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.193367 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.193268 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-workload-socket\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.193367 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.193329 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-credential-socket\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.193534 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.193513 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/116dd55c-1850-4767-8d20-3760e20ea330-istiod-ca-cert\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.195232 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.195212 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-istio-envoy\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.195340 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.195322 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/116dd55c-1850-4767-8d20-3760e20ea330-istio-podinfo\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.204485 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.204459 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vpql\" (UniqueName: \"kubernetes.io/projected/116dd55c-1850-4767-8d20-3760e20ea330-kube-api-access-8vpql\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.204637 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.204511 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/116dd55c-1850-4767-8d20-3760e20ea330-istio-token\") pod \"e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.292301 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.292258 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:40.440736 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.440705 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg"] Jul 14 13:52:40.442829 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:52:40.442805 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod116dd55c_1850_4767_8d20_3760e20ea330.slice/crio-3761c119c5418785ab2eb5c776119e98237a8db61d9568d1a738026f637e6a1e WatchSource:0}: Error finding container 3761c119c5418785ab2eb5c776119e98237a8db61d9568d1a738026f637e6a1e: Status 404 returned error can't find the container with id 3761c119c5418785ab2eb5c776119e98237a8db61d9568d1a738026f637e6a1e Jul 14 13:52:40.444962 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.444930 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:52:40.445037 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.444995 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:52:40.445037 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.445024 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:52:40.722353 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.722251 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" event={"ID":"116dd55c-1850-4767-8d20-3760e20ea330","Type":"ContainerStarted","Data":"87bf98bda21ae7dae85717965f991ee47f724287d4b82511917ba9f7bb5c6902"} Jul 14 13:52:40.722353 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.722305 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" event={"ID":"116dd55c-1850-4767-8d20-3760e20ea330","Type":"ContainerStarted","Data":"3761c119c5418785ab2eb5c776119e98237a8db61d9568d1a738026f637e6a1e"} Jul 14 13:52:40.752768 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:40.752715 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" podStartSLOduration=1.75269661 podStartE2EDuration="1.75269661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:52:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:52:40.752336595 +0000 UTC m=+1524.279487230" watchObservedRunningTime="2026-07-14 13:52:40.75269661 +0000 UTC m=+1524.279847246" Jul 14 13:52:41.177967 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:41.177931 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:52:41.279901 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:41.279867 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:52:41.292854 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:41.292823 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:41.298138 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:41.298113 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:41.378642 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:41.378610 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:52:41.727725 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:41.727684 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:52:41.727924 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:41.727797 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:41.728913 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:41.728890 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:42.045750 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:42.045716 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:52:42.142810 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:42.142776 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:52:44.325590 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:44.325555 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:52:57.540284 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:57.540217 2583 patch_prober.go:28] interesting pod/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": read tcp 10.134.0.2:51102->10.134.0.41:15021: read: connection reset by peer" start-of-body= Jul 14 13:52:57.540979 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:57.540313 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" podUID="116dd55c-1850-4767-8d20-3760e20ea330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": read tcp 10.134.0.2:51102->10.134.0.41:15021: read: connection reset by peer" Jul 14 13:52:57.807891 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:57.807817 2583 generic.go:356] "Generic (PLEG): container finished" podID="116dd55c-1850-4767-8d20-3760e20ea330" containerID="87bf98bda21ae7dae85717965f991ee47f724287d4b82511917ba9f7bb5c6902" exitCode=0 Jul 14 13:52:57.808029 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:57.807893 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" event={"ID":"116dd55c-1850-4767-8d20-3760e20ea330","Type":"ContainerDied","Data":"87bf98bda21ae7dae85717965f991ee47f724287d4b82511917ba9f7bb5c6902"} Jul 14 13:52:57.808337 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:57.808320 2583 scope.go:117] "RemoveContainer" containerID="87bf98bda21ae7dae85717965f991ee47f724287d4b82511917ba9f7bb5c6902" Jul 14 13:52:57.809390 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:57.809359 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:52:57.809484 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:57.809432 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:52:57.809484 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:57.809465 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:52:58.814340 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:58.814304 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" event={"ID":"116dd55c-1850-4767-8d20-3760e20ea330","Type":"ContainerStarted","Data":"5fdeff26253e3d01a415c1e62a65cc00af49cc1bae79651a6e820a44acee9be9"} Jul 14 13:52:59.292850 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:59.292813 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:52:59.293113 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:59.293060 2583 patch_prober.go:28] interesting pod/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 14 13:52:59.293216 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:52:59.293149 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" podUID="116dd55c-1850-4767-8d20-3760e20ea330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 14 13:53:00.292894 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:53:00.292859 2583 patch_prober.go:28] interesting pod/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 14 13:53:00.293311 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:53:00.292916 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" podUID="116dd55c-1850-4767-8d20-3760e20ea330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 14 13:53:01.293166 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:53:01.293126 2583 patch_prober.go:28] interesting pod/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 14 13:53:01.293545 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:53:01.293198 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" podUID="116dd55c-1850-4767-8d20-3760e20ea330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 14 13:53:02.292822 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:53:02.292786 2583 patch_prober.go:28] interesting pod/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 14 13:53:02.292991 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:53:02.292846 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" podUID="116dd55c-1850-4767-8d20-3760e20ea330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 14 13:53:03.296711 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:53:03.296680 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:53:03.297097 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:53:03.297043 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:53:03.297623 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:53:03.297603 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:53:40.941969 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:53:40.941936 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:53:46.689912 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:53:46.689874 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:53:51.329287 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:53:51.329253 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:53:57.823635 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:53:57.823594 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:53:57.923419 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:53:57.923382 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:54:01.558020 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:01.557983 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg"] Jul 14 13:54:01.558490 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:01.558198 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" podUID="116dd55c-1850-4767-8d20-3760e20ea330" containerName="istio-proxy" containerID="cri-o://5fdeff26253e3d01a415c1e62a65cc00af49cc1bae79651a6e820a44acee9be9" gracePeriod=30 Jul 14 13:54:01.592391 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:01.592356 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:54:01.992490 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:01.992404 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:54:02.111485 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.111446 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq"] Jul 14 13:54:02.115338 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.115312 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.118183 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.118157 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-0df6d38a-gw-openshift-default-dockercfg-c45cn\"" Jul 14 13:54:02.119520 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.119482 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:54:02.128443 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.128365 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq"] Jul 14 13:54:02.160022 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.159984 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-workload-certs\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.160205 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.160040 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-envoy\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.160205 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.160064 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-data\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.160205 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.160129 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istiod-ca-cert\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.160205 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.160196 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6r4t\" (UniqueName: \"kubernetes.io/projected/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-kube-api-access-f6r4t\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.160347 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.160258 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-podinfo\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.160347 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.160293 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-credential-socket\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.160347 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.160317 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-token\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.160347 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.160338 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-workload-socket\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.233798 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.233753 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:54:02.260772 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.260735 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6r4t\" (UniqueName: \"kubernetes.io/projected/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-kube-api-access-f6r4t\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.260977 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.260800 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-podinfo\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.260977 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.260831 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-credential-socket\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.260977 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.260860 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-token\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.260977 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.260888 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-workload-socket\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.260977 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.260931 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-workload-certs\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.261282 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.260990 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-envoy\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.261282 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.261017 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-data\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.261282 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.261045 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istiod-ca-cert\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.261453 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.261371 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-credential-socket\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.261453 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.261410 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-workload-socket\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.261453 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.261433 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-workload-certs\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.261608 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.261494 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-data\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.261862 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.261841 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istiod-ca-cert\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.263357 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.263338 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-envoy\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.263602 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.263585 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-podinfo\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.270226 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.270193 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-token\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.270610 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.270587 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6r4t\" (UniqueName: \"kubernetes.io/projected/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-kube-api-access-f6r4t\") pod \"e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.432849 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.432790 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:02.579757 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.579731 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq"] Jul 14 13:54:02.581855 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:54:02.581826 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd2c5eb1_4356_4a82_9f9f_c3548bde56e6.slice/crio-3e6f27b35ffbf59dc852ce4df174df6549cde0674ad837056cbdd085a4b01cec WatchSource:0}: Error finding container 3e6f27b35ffbf59dc852ce4df174df6549cde0674ad837056cbdd085a4b01cec: Status 404 returned error can't find the container with id 3e6f27b35ffbf59dc852ce4df174df6549cde0674ad837056cbdd085a4b01cec Jul 14 13:54:02.584024 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.583987 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:54:02.584112 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.584052 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:54:02.584159 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:02.584112 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:54:03.138749 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:03.138715 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" event={"ID":"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6","Type":"ContainerStarted","Data":"cf10a04dd3ed3dcff328bfa5a0ba54fd8345a77dc08efe5bc85fc6896f5826bf"} Jul 14 13:54:03.138927 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:03.138770 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" event={"ID":"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6","Type":"ContainerStarted","Data":"3e6f27b35ffbf59dc852ce4df174df6549cde0674ad837056cbdd085a4b01cec"} Jul 14 13:54:03.161799 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:03.161755 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" podStartSLOduration=1.161737634 podStartE2EDuration="1.161737634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:54:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:03.161251675 +0000 UTC m=+1606.688402322" watchObservedRunningTime="2026-07-14 13:54:03.161737634 +0000 UTC m=+1606.688888274" Jul 14 13:54:03.299129 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:03.299076 2583 patch_prober.go:28] interesting pod/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 13:54:03.299291 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:03.299147 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" podUID="116dd55c-1850-4767-8d20-3760e20ea330" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 13:54:03.384048 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:03.384011 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:54:03.433064 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:03.432979 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:03.439997 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:03.439969 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:03.758557 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:03.758520 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:54:04.026704 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:04.026596 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:54:04.132769 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:04.125008 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:54:04.143696 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:04.143663 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:04.144654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:04.144635 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:04.226329 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:04.226291 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:54:06.826590 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.826511 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:54:06.906879 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.906850 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vpql\" (UniqueName: \"kubernetes.io/projected/116dd55c-1850-4767-8d20-3760e20ea330-kube-api-access-8vpql\") pod \"116dd55c-1850-4767-8d20-3760e20ea330\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " Jul 14 13:54:06.907044 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.906917 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/116dd55c-1850-4767-8d20-3760e20ea330-istiod-ca-cert\") pod \"116dd55c-1850-4767-8d20-3760e20ea330\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " Jul 14 13:54:06.907044 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.907013 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-workload-socket\") pod \"116dd55c-1850-4767-8d20-3760e20ea330\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " Jul 14 13:54:06.907174 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.907104 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-credential-socket\") pod \"116dd55c-1850-4767-8d20-3760e20ea330\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " Jul 14 13:54:06.907174 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.907142 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-istio-envoy\") pod \"116dd55c-1850-4767-8d20-3760e20ea330\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " Jul 14 13:54:06.907280 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.907177 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-istio-data\") pod \"116dd55c-1850-4767-8d20-3760e20ea330\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " Jul 14 13:54:06.907280 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.907244 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-workload-certs\") pod \"116dd55c-1850-4767-8d20-3760e20ea330\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " Jul 14 13:54:06.907280 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.907277 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/116dd55c-1850-4767-8d20-3760e20ea330-istio-podinfo\") pod \"116dd55c-1850-4767-8d20-3760e20ea330\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " Jul 14 13:54:06.907437 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.907277 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "116dd55c-1850-4767-8d20-3760e20ea330" (UID: "116dd55c-1850-4767-8d20-3760e20ea330"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:54:06.907437 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.907310 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "116dd55c-1850-4767-8d20-3760e20ea330" (UID: "116dd55c-1850-4767-8d20-3760e20ea330"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:54:06.907437 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.907320 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/116dd55c-1850-4767-8d20-3760e20ea330-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "116dd55c-1850-4767-8d20-3760e20ea330" (UID: "116dd55c-1850-4767-8d20-3760e20ea330"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:54:06.907437 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.907352 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/116dd55c-1850-4767-8d20-3760e20ea330-istio-token\") pod \"116dd55c-1850-4767-8d20-3760e20ea330\" (UID: \"116dd55c-1850-4767-8d20-3760e20ea330\") " Jul 14 13:54:06.907650 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.907571 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "116dd55c-1850-4767-8d20-3760e20ea330" (UID: "116dd55c-1850-4767-8d20-3760e20ea330"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:54:06.907707 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.907678 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/116dd55c-1850-4767-8d20-3760e20ea330-istiod-ca-cert\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:54:06.907707 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.907697 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-workload-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:54:06.907807 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.907710 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-credential-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:54:06.907807 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.907726 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-workload-certs\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:54:06.908037 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.908001 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-istio-data" (OuterVolumeSpecName: "istio-data") pod "116dd55c-1850-4767-8d20-3760e20ea330" (UID: "116dd55c-1850-4767-8d20-3760e20ea330"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:54:06.909542 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.909517 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "116dd55c-1850-4767-8d20-3760e20ea330" (UID: "116dd55c-1850-4767-8d20-3760e20ea330"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:54:06.909542 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.909531 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/116dd55c-1850-4767-8d20-3760e20ea330-kube-api-access-8vpql" (OuterVolumeSpecName: "kube-api-access-8vpql") pod "116dd55c-1850-4767-8d20-3760e20ea330" (UID: "116dd55c-1850-4767-8d20-3760e20ea330"). InnerVolumeSpecName "kube-api-access-8vpql". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:54:06.909779 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.909756 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/116dd55c-1850-4767-8d20-3760e20ea330-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "116dd55c-1850-4767-8d20-3760e20ea330" (UID: "116dd55c-1850-4767-8d20-3760e20ea330"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 13:54:06.910002 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:06.909987 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/116dd55c-1850-4767-8d20-3760e20ea330-istio-token" (OuterVolumeSpecName: "istio-token") pod "116dd55c-1850-4767-8d20-3760e20ea330" (UID: "116dd55c-1850-4767-8d20-3760e20ea330"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:54:07.008708 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:07.008663 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-istio-envoy\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:54:07.008708 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:07.008700 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/116dd55c-1850-4767-8d20-3760e20ea330-istio-data\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:54:07.008708 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:07.008714 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/116dd55c-1850-4767-8d20-3760e20ea330-istio-podinfo\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:54:07.008962 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:07.008727 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/116dd55c-1850-4767-8d20-3760e20ea330-istio-token\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:54:07.008962 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:07.008740 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vpql\" (UniqueName: \"kubernetes.io/projected/116dd55c-1850-4767-8d20-3760e20ea330-kube-api-access-8vpql\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:54:07.159535 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:07.159445 2583 generic.go:356] "Generic (PLEG): container finished" podID="116dd55c-1850-4767-8d20-3760e20ea330" containerID="5fdeff26253e3d01a415c1e62a65cc00af49cc1bae79651a6e820a44acee9be9" exitCode=0 Jul 14 13:54:07.159535 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:07.159496 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" Jul 14 13:54:07.159535 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:07.159504 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" event={"ID":"116dd55c-1850-4767-8d20-3760e20ea330","Type":"ContainerDied","Data":"5fdeff26253e3d01a415c1e62a65cc00af49cc1bae79651a6e820a44acee9be9"} Jul 14 13:54:07.159771 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:07.159554 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg" event={"ID":"116dd55c-1850-4767-8d20-3760e20ea330","Type":"ContainerDied","Data":"3761c119c5418785ab2eb5c776119e98237a8db61d9568d1a738026f637e6a1e"} Jul 14 13:54:07.159771 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:07.159580 2583 scope.go:117] "RemoveContainer" containerID="5fdeff26253e3d01a415c1e62a65cc00af49cc1bae79651a6e820a44acee9be9" Jul 14 13:54:07.169362 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:07.169334 2583 scope.go:117] "RemoveContainer" containerID="87bf98bda21ae7dae85717965f991ee47f724287d4b82511917ba9f7bb5c6902" Jul 14 13:54:07.178048 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:07.178027 2583 scope.go:117] "RemoveContainer" containerID="5fdeff26253e3d01a415c1e62a65cc00af49cc1bae79651a6e820a44acee9be9" Jul 14 13:54:07.178369 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:54:07.178345 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fdeff26253e3d01a415c1e62a65cc00af49cc1bae79651a6e820a44acee9be9\": container with ID starting with 5fdeff26253e3d01a415c1e62a65cc00af49cc1bae79651a6e820a44acee9be9 not found: ID does not exist" containerID="5fdeff26253e3d01a415c1e62a65cc00af49cc1bae79651a6e820a44acee9be9" Jul 14 13:54:07.178439 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:07.178378 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fdeff26253e3d01a415c1e62a65cc00af49cc1bae79651a6e820a44acee9be9"} err="failed to get container status \"5fdeff26253e3d01a415c1e62a65cc00af49cc1bae79651a6e820a44acee9be9\": rpc error: code = NotFound desc = could not find container \"5fdeff26253e3d01a415c1e62a65cc00af49cc1bae79651a6e820a44acee9be9\": container with ID starting with 5fdeff26253e3d01a415c1e62a65cc00af49cc1bae79651a6e820a44acee9be9 not found: ID does not exist" Jul 14 13:54:07.178439 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:07.178399 2583 scope.go:117] "RemoveContainer" containerID="87bf98bda21ae7dae85717965f991ee47f724287d4b82511917ba9f7bb5c6902" Jul 14 13:54:07.178623 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:54:07.178607 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87bf98bda21ae7dae85717965f991ee47f724287d4b82511917ba9f7bb5c6902\": container with ID starting with 87bf98bda21ae7dae85717965f991ee47f724287d4b82511917ba9f7bb5c6902 not found: ID does not exist" containerID="87bf98bda21ae7dae85717965f991ee47f724287d4b82511917ba9f7bb5c6902" Jul 14 13:54:07.178670 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:07.178627 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87bf98bda21ae7dae85717965f991ee47f724287d4b82511917ba9f7bb5c6902"} err="failed to get container status \"87bf98bda21ae7dae85717965f991ee47f724287d4b82511917ba9f7bb5c6902\": rpc error: code = NotFound desc = could not find container \"87bf98bda21ae7dae85717965f991ee47f724287d4b82511917ba9f7bb5c6902\": container with ID starting with 87bf98bda21ae7dae85717965f991ee47f724287d4b82511917ba9f7bb5c6902 not found: ID does not exist" Jul 14 13:54:07.185194 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:07.185166 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg"] Jul 14 13:54:07.189727 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:07.189696 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b0840d76-openshift-default-677fb8b997-rfpjg"] Jul 14 13:54:08.290613 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:08.290558 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:54:08.399456 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:08.399420 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:54:08.494058 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:08.494020 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="116dd55c-1850-4767-8d20-3760e20ea330" path="/var/lib/kubelet/pods/116dd55c-1850-4767-8d20-3760e20ea330/volumes" Jul 14 13:54:11.756875 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:11.756841 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:54:19.164722 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:19.164070 2583 patch_prober.go:28] interesting pod/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": read tcp 10.134.0.2:54722->10.134.0.42:15021: read: connection reset by peer" start-of-body= Jul 14 13:54:19.164722 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:19.164156 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" podUID="cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": read tcp 10.134.0.2:54722->10.134.0.42:15021: read: connection reset by peer" Jul 14 13:54:19.220952 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:19.220917 2583 generic.go:356] "Generic (PLEG): container finished" podID="cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" containerID="cf10a04dd3ed3dcff328bfa5a0ba54fd8345a77dc08efe5bc85fc6896f5826bf" exitCode=0 Jul 14 13:54:19.221134 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:19.220997 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" event={"ID":"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6","Type":"ContainerDied","Data":"cf10a04dd3ed3dcff328bfa5a0ba54fd8345a77dc08efe5bc85fc6896f5826bf"} Jul 14 13:54:19.221470 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:19.221456 2583 scope.go:117] "RemoveContainer" containerID="cf10a04dd3ed3dcff328bfa5a0ba54fd8345a77dc08efe5bc85fc6896f5826bf" Jul 14 13:54:19.222545 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:19.222518 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:54:19.222643 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:19.222600 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:54:19.222703 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:19.222651 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:54:20.230020 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:20.229975 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" event={"ID":"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6","Type":"ContainerStarted","Data":"098048b7b96a44810c59d3cc6fa6369747b0f1cfacf8573891628aefbdefae9c"} Jul 14 13:54:20.433423 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:20.433387 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:20.433767 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:20.433685 2583 patch_prober.go:28] interesting pod/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 14 13:54:20.433953 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:20.433931 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" podUID="cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 14 13:54:21.434358 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:21.434317 2583 patch_prober.go:28] interesting pod/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 14 13:54:21.434869 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:21.434389 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" podUID="cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 14 13:54:22.434269 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:22.434224 2583 patch_prober.go:28] interesting pod/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 14 13:54:22.434470 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:22.434296 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" podUID="cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 14 13:54:23.433846 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:23.433800 2583 patch_prober.go:28] interesting pod/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 14 13:54:23.434064 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:23.433877 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" podUID="cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 14 13:54:24.437276 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:24.437243 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:24.437718 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:24.437608 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:54:24.438317 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:54:24.438296 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:55:04.414194 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:04.414146 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:55:08.049558 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:08.049518 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq"] Jul 14 13:55:08.050009 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:08.049765 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" podUID="cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" containerName="istio-proxy" containerID="cri-o://098048b7b96a44810c59d3cc6fa6369747b0f1cfacf8573891628aefbdefae9c" gracePeriod=30 Jul 14 13:55:08.091152 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:08.091113 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:55:08.201320 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:08.201268 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:55:08.718700 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:08.718660 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:55:09.439807 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:09.439769 2583 patch_prober.go:28] interesting pod/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 13:55:09.440204 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:09.439829 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" podUID="cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 13:55:13.312272 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.312243 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:55:13.352651 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.352550 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-envoy\") pod \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " Jul 14 13:55:13.352651 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.352597 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6r4t\" (UniqueName: \"kubernetes.io/projected/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-kube-api-access-f6r4t\") pod \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " Jul 14 13:55:13.352651 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.352613 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-credential-socket\") pod \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " Jul 14 13:55:13.352651 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.352629 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-workload-socket\") pod \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " Jul 14 13:55:13.352989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.352670 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istiod-ca-cert\") pod \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " Jul 14 13:55:13.352989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.352710 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-workload-certs\") pod \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " Jul 14 13:55:13.352989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.352774 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-data\") pod \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " Jul 14 13:55:13.352989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.352832 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-token\") pod \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " Jul 14 13:55:13.352989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.352887 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-podinfo\") pod \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\" (UID: \"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6\") " Jul 14 13:55:13.352989 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.352982 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" (UID: "cd2c5eb1-4356-4a82-9f9f-c3548bde56e6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:55:13.353348 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.353214 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-credential-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:55:13.353555 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.353517 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" (UID: "cd2c5eb1-4356-4a82-9f9f-c3548bde56e6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:55:13.353734 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.353707 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" (UID: "cd2c5eb1-4356-4a82-9f9f-c3548bde56e6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:55:13.354032 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.353996 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" (UID: "cd2c5eb1-4356-4a82-9f9f-c3548bde56e6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:55:13.354434 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.354390 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-data" (OuterVolumeSpecName: "istio-data") pod "cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" (UID: "cd2c5eb1-4356-4a82-9f9f-c3548bde56e6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:55:13.355421 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.355384 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" (UID: "cd2c5eb1-4356-4a82-9f9f-c3548bde56e6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:55:13.355503 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.355452 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" (UID: "cd2c5eb1-4356-4a82-9f9f-c3548bde56e6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 13:55:13.355794 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.355770 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-token" (OuterVolumeSpecName: "istio-token") pod "cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" (UID: "cd2c5eb1-4356-4a82-9f9f-c3548bde56e6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:55:13.355862 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.355824 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-kube-api-access-f6r4t" (OuterVolumeSpecName: "kube-api-access-f6r4t") pod "cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" (UID: "cd2c5eb1-4356-4a82-9f9f-c3548bde56e6"). InnerVolumeSpecName "kube-api-access-f6r4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:55:13.454569 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.454533 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-data\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:55:13.454569 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.454564 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-token\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:55:13.454569 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.454573 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-podinfo\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:55:13.454808 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.454582 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istio-envoy\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:55:13.454808 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.454591 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6r4t\" (UniqueName: \"kubernetes.io/projected/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-kube-api-access-f6r4t\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:55:13.454808 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.454600 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-workload-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:55:13.454808 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.454609 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-istiod-ca-cert\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:55:13.454808 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.454617 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6-workload-certs\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:55:13.498409 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.498374 2583 generic.go:356] "Generic (PLEG): container finished" podID="cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" containerID="098048b7b96a44810c59d3cc6fa6369747b0f1cfacf8573891628aefbdefae9c" exitCode=0 Jul 14 13:55:13.498576 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.498419 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" Jul 14 13:55:13.498576 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.498450 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" event={"ID":"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6","Type":"ContainerDied","Data":"098048b7b96a44810c59d3cc6fa6369747b0f1cfacf8573891628aefbdefae9c"} Jul 14 13:55:13.498576 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.498501 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq" event={"ID":"cd2c5eb1-4356-4a82-9f9f-c3548bde56e6","Type":"ContainerDied","Data":"3e6f27b35ffbf59dc852ce4df174df6549cde0674ad837056cbdd085a4b01cec"} Jul 14 13:55:13.498576 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.498526 2583 scope.go:117] "RemoveContainer" containerID="098048b7b96a44810c59d3cc6fa6369747b0f1cfacf8573891628aefbdefae9c" Jul 14 13:55:13.508288 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.508262 2583 scope.go:117] "RemoveContainer" containerID="cf10a04dd3ed3dcff328bfa5a0ba54fd8345a77dc08efe5bc85fc6896f5826bf" Jul 14 13:55:13.517295 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.517262 2583 scope.go:117] "RemoveContainer" containerID="098048b7b96a44810c59d3cc6fa6369747b0f1cfacf8573891628aefbdefae9c" Jul 14 13:55:13.517594 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:55:13.517577 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"098048b7b96a44810c59d3cc6fa6369747b0f1cfacf8573891628aefbdefae9c\": container with ID starting with 098048b7b96a44810c59d3cc6fa6369747b0f1cfacf8573891628aefbdefae9c not found: ID does not exist" containerID="098048b7b96a44810c59d3cc6fa6369747b0f1cfacf8573891628aefbdefae9c" Jul 14 13:55:13.517673 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.517610 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"098048b7b96a44810c59d3cc6fa6369747b0f1cfacf8573891628aefbdefae9c"} err="failed to get container status \"098048b7b96a44810c59d3cc6fa6369747b0f1cfacf8573891628aefbdefae9c\": rpc error: code = NotFound desc = could not find container \"098048b7b96a44810c59d3cc6fa6369747b0f1cfacf8573891628aefbdefae9c\": container with ID starting with 098048b7b96a44810c59d3cc6fa6369747b0f1cfacf8573891628aefbdefae9c not found: ID does not exist" Jul 14 13:55:13.517673 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.517636 2583 scope.go:117] "RemoveContainer" containerID="cf10a04dd3ed3dcff328bfa5a0ba54fd8345a77dc08efe5bc85fc6896f5826bf" Jul 14 13:55:13.517928 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:55:13.517912 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf10a04dd3ed3dcff328bfa5a0ba54fd8345a77dc08efe5bc85fc6896f5826bf\": container with ID starting with cf10a04dd3ed3dcff328bfa5a0ba54fd8345a77dc08efe5bc85fc6896f5826bf not found: ID does not exist" containerID="cf10a04dd3ed3dcff328bfa5a0ba54fd8345a77dc08efe5bc85fc6896f5826bf" Jul 14 13:55:13.517968 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.517933 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf10a04dd3ed3dcff328bfa5a0ba54fd8345a77dc08efe5bc85fc6896f5826bf"} err="failed to get container status \"cf10a04dd3ed3dcff328bfa5a0ba54fd8345a77dc08efe5bc85fc6896f5826bf\": rpc error: code = NotFound desc = could not find container \"cf10a04dd3ed3dcff328bfa5a0ba54fd8345a77dc08efe5bc85fc6896f5826bf\": container with ID starting with cf10a04dd3ed3dcff328bfa5a0ba54fd8345a77dc08efe5bc85fc6896f5826bf not found: ID does not exist" Jul 14 13:55:13.523236 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.523192 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq"] Jul 14 13:55:13.528024 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:13.527999 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-0df6d38a-gw-openshift-default-564f7dfdc-k8xpq"] Jul 14 13:55:14.494859 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.494808 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" path="/var/lib/kubelet/pods/cd2c5eb1-4356-4a82-9f9f-c3548bde56e6/volumes" Jul 14 13:55:14.527406 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.527372 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:55:14.628198 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.628153 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:55:14.711145 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.711110 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q"] Jul 14 13:55:14.711693 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.711671 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="116dd55c-1850-4767-8d20-3760e20ea330" containerName="istio-proxy" Jul 14 13:55:14.711693 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.711694 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="116dd55c-1850-4767-8d20-3760e20ea330" containerName="istio-proxy" Jul 14 13:55:14.711844 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.711714 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" containerName="istio-proxy" Jul 14 13:55:14.711844 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.711722 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" containerName="istio-proxy" Jul 14 13:55:14.711844 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.711741 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="116dd55c-1850-4767-8d20-3760e20ea330" containerName="istio-proxy" Jul 14 13:55:14.711844 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.711751 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="116dd55c-1850-4767-8d20-3760e20ea330" containerName="istio-proxy" Jul 14 13:55:14.711844 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.711841 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="116dd55c-1850-4767-8d20-3760e20ea330" containerName="istio-proxy" Jul 14 13:55:14.712113 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.711855 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="116dd55c-1850-4767-8d20-3760e20ea330" containerName="istio-proxy" Jul 14 13:55:14.712113 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.711865 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" containerName="istio-proxy" Jul 14 13:55:14.712113 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.711874 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" containerName="istio-proxy" Jul 14 13:55:14.717007 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.716983 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.719515 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.719495 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b3b6f94a-openshift-default-dockercfg-n5mq5\"" Jul 14 13:55:14.728200 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.728170 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q"] Jul 14 13:55:14.750495 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.750463 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:55:14.767595 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.767555 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-workload-certs\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.767766 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.767614 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shc45\" (UniqueName: \"kubernetes.io/projected/d6529521-49b4-4e6d-8b0b-959bed635a79-kube-api-access-shc45\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.767766 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.767673 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-envoy\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.767766 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.767692 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-data\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.767875 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.767769 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-podinfo\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.767875 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.767825 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6529521-49b4-4e6d-8b0b-959bed635a79-istiod-ca-cert\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.767944 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.767872 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-workload-socket\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.767944 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.767901 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-token\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.767944 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.767935 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-credential-socket\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.830753 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.830715 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:55:14.868561 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.868521 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-envoy\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.868561 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.868561 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-data\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.868774 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.868696 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-podinfo\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.868774 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.868750 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6529521-49b4-4e6d-8b0b-959bed635a79-istiod-ca-cert\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.868846 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.868790 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-workload-socket\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.868846 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.868808 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-token\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.868846 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.868840 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-credential-socket\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.868996 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.868891 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-workload-certs\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.868996 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.868947 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shc45\" (UniqueName: \"kubernetes.io/projected/d6529521-49b4-4e6d-8b0b-959bed635a79-kube-api-access-shc45\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.868996 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.868958 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-data\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.869292 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.869245 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-credential-socket\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.869292 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.869268 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-workload-certs\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.869453 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.869324 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-workload-socket\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.869589 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.869567 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6529521-49b4-4e6d-8b0b-959bed635a79-istiod-ca-cert\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.871042 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.871014 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-envoy\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.871340 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.871324 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-podinfo\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.878318 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.878292 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shc45\" (UniqueName: \"kubernetes.io/projected/d6529521-49b4-4e6d-8b0b-959bed635a79-kube-api-access-shc45\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:14.878425 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:14.878342 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-token\") pod \"e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:15.034942 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:15.034853 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:15.179599 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:15.179572 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q"] Jul 14 13:55:15.181689 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:55:15.181663 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6529521_49b4_4e6d_8b0b_959bed635a79.slice/crio-4f171d68c5a29d522749388e61ca758cc48bbc5275599a50750ca7b47634d9c2 WatchSource:0}: Error finding container 4f171d68c5a29d522749388e61ca758cc48bbc5275599a50750ca7b47634d9c2: Status 404 returned error can't find the container with id 4f171d68c5a29d522749388e61ca758cc48bbc5275599a50750ca7b47634d9c2 Jul 14 13:55:15.183838 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:15.183807 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:55:15.183930 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:15.183892 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:55:15.183987 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:15.183938 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:55:15.514582 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:15.514545 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" event={"ID":"d6529521-49b4-4e6d-8b0b-959bed635a79","Type":"ContainerStarted","Data":"9137dfadffaeb5d4d86e2994783726cc9aa48ade95c480c8bc3080682dab8464"} Jul 14 13:55:15.514958 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:15.514608 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" event={"ID":"d6529521-49b4-4e6d-8b0b-959bed635a79","Type":"ContainerStarted","Data":"4f171d68c5a29d522749388e61ca758cc48bbc5275599a50750ca7b47634d9c2"} Jul 14 13:55:15.535211 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:15.535147 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" podStartSLOduration=1.535127533 podStartE2EDuration="1.535127533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:55:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:15.534912993 +0000 UTC m=+1679.062063629" watchObservedRunningTime="2026-07-14 13:55:15.535127533 +0000 UTC m=+1679.062278169" Jul 14 13:55:15.849557 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:15.849448 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:55:15.954299 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:15.954260 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:55:16.035991 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:16.035955 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:16.041739 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:16.041709 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:16.117954 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:16.117829 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:55:16.338907 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:16.338874 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:55:16.503373 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:16.503343 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:55:16.520118 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:16.520061 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:16.521091 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:16.521061 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:16.601390 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:16.601358 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:55:18.071739 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:18.071673 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:55:32.562284 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:32.562239 2583 patch_prober.go:28] interesting pod/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 14 13:55:32.562729 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:32.562321 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" podUID="d6529521-49b4-4e6d-8b0b-959bed635a79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 14 13:55:33.609330 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:33.609287 2583 generic.go:356] "Generic (PLEG): container finished" podID="d6529521-49b4-4e6d-8b0b-959bed635a79" containerID="9137dfadffaeb5d4d86e2994783726cc9aa48ade95c480c8bc3080682dab8464" exitCode=0 Jul 14 13:55:33.609943 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:33.609367 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" event={"ID":"d6529521-49b4-4e6d-8b0b-959bed635a79","Type":"ContainerDied","Data":"9137dfadffaeb5d4d86e2994783726cc9aa48ade95c480c8bc3080682dab8464"} Jul 14 13:55:33.609943 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:33.609793 2583 scope.go:117] "RemoveContainer" containerID="9137dfadffaeb5d4d86e2994783726cc9aa48ade95c480c8bc3080682dab8464" Jul 14 13:55:33.610794 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:33.610766 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:55:33.610909 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:33.610844 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:55:33.610972 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:33.610916 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:55:34.615592 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:34.615554 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" event={"ID":"d6529521-49b4-4e6d-8b0b-959bed635a79","Type":"ContainerStarted","Data":"a0991abba1ffe77a3fa42f5ec3ffdde05ef33d365800e89fa295c8b165200174"} Jul 14 13:55:35.036358 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:35.035818 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:35.036358 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:35.035992 2583 patch_prober.go:28] interesting pod/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 14 13:55:35.036358 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:35.036050 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" podUID="d6529521-49b4-4e6d-8b0b-959bed635a79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 14 13:55:36.036265 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:36.036215 2583 patch_prober.go:28] interesting pod/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 14 13:55:36.036791 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:36.036308 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" podUID="d6529521-49b4-4e6d-8b0b-959bed635a79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 14 13:55:37.035792 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:37.035751 2583 patch_prober.go:28] interesting pod/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 14 13:55:37.036180 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:37.036136 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" podUID="d6529521-49b4-4e6d-8b0b-959bed635a79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 14 13:55:38.035967 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:38.035923 2583 patch_prober.go:28] interesting pod/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 14 13:55:38.036532 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:38.036002 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" podUID="d6529521-49b4-4e6d-8b0b-959bed635a79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 14 13:55:39.039627 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:39.039591 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:39.040154 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:39.040113 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:39.041046 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:39.041026 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:55:41.159484 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:41.159444 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:55:41.259696 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:41.259659 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:55:51.640839 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:51.640803 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:55:51.740820 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:55:51.740788 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:56:31.508596 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:31.508552 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:56:36.083061 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:36.083023 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:56:42.403712 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:42.403667 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:56:42.506379 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:42.506309 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:56:47.177905 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.177865 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q"] Jul 14 13:56:47.178512 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.178183 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" podUID="d6529521-49b4-4e6d-8b0b-959bed635a79" containerName="istio-proxy" containerID="cri-o://a0991abba1ffe77a3fa42f5ec3ffdde05ef33d365800e89fa295c8b165200174" gracePeriod=30 Jul 14 13:56:47.214465 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.214412 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:56:47.494513 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.494418 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj"] Jul 14 13:56:47.494897 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.494883 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" containerName="istio-proxy" Jul 14 13:56:47.494941 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.494899 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd2c5eb1-4356-4a82-9f9f-c3548bde56e6" containerName="istio-proxy" Jul 14 13:56:47.498349 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.498323 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.503986 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.503962 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-fe2b25e2-openshift-default-dockercfg-c78vf\"" Jul 14 13:56:47.511688 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.511658 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj"] Jul 14 13:56:47.527240 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.527207 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-credential-socket\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.527383 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.527267 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkwg7\" (UniqueName: \"kubernetes.io/projected/a85bdc03-fd2a-442f-9388-881cc2448b56-kube-api-access-tkwg7\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.527383 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.527311 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-envoy\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.527383 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.527368 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-workload-socket\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.527552 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.527397 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a85bdc03-fd2a-442f-9388-881cc2448b56-istiod-ca-cert\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.527552 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.527470 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-workload-certs\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.527552 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.527531 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-podinfo\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.527706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.527559 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-data\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.527706 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.527590 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-token\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.532984 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.532955 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:56:47.628249 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.628212 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-data\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.628249 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.628252 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-token\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.628494 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.628297 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-credential-socket\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.628494 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.628338 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkwg7\" (UniqueName: \"kubernetes.io/projected/a85bdc03-fd2a-442f-9388-881cc2448b56-kube-api-access-tkwg7\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.628494 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.628379 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-envoy\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.628494 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.628434 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-workload-socket\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.628494 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.628481 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a85bdc03-fd2a-442f-9388-881cc2448b56-istiod-ca-cert\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.628780 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.628539 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-workload-certs\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.628780 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.628591 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-podinfo\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.628780 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.628653 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-data\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.629286 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.629258 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-credential-socket\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.629409 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.629333 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-workload-socket\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.629409 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.629384 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a85bdc03-fd2a-442f-9388-881cc2448b56-istiod-ca-cert\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.629619 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.629594 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-workload-certs\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.631160 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.631138 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-podinfo\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.631266 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.631142 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-envoy\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.638175 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.638153 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkwg7\" (UniqueName: \"kubernetes.io/projected/a85bdc03-fd2a-442f-9388-881cc2448b56-kube-api-access-tkwg7\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.638408 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.638388 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-token\") pod \"e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.811071 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.811026 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:47.955780 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.955754 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj"] Jul 14 13:56:47.957956 ip-10-0-142-94 kubenswrapper[2583]: W0714 13:56:47.957919 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda85bdc03_fd2a_442f_9388_881cc2448b56.slice/crio-4f241c235f2298186dca3d92046eb934fe283263a5eebe07ac5f3a996dba3cd2 WatchSource:0}: Error finding container 4f241c235f2298186dca3d92046eb934fe283263a5eebe07ac5f3a996dba3cd2: Status 404 returned error can't find the container with id 4f241c235f2298186dca3d92046eb934fe283263a5eebe07ac5f3a996dba3cd2 Jul 14 13:56:47.960654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.960623 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:56:47.960775 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.960703 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:56:47.960775 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.960750 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:56:47.983970 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:47.983946 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" event={"ID":"a85bdc03-fd2a-442f-9388-881cc2448b56","Type":"ContainerStarted","Data":"4f241c235f2298186dca3d92046eb934fe283263a5eebe07ac5f3a996dba3cd2"} Jul 14 13:56:48.631136 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:48.631070 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:56:48.711654 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:48.711609 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:56:48.806622 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:48.806590 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:56:48.973589 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:48.973491 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:56:48.998521 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:48.998481 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" event={"ID":"a85bdc03-fd2a-442f-9388-881cc2448b56","Type":"ContainerStarted","Data":"a1c715f302bb7af4099c173f4a8e84ed37be91eb7f078dd4887b47afe44aa273"} Jul 14 13:56:49.031114 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:49.031038 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" podStartSLOduration=2.031017422 podStartE2EDuration="2.031017422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:56:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:56:49.02780704 +0000 UTC m=+1772.554957677" watchObservedRunningTime="2026-07-14 13:56:49.031017422 +0000 UTC m=+1772.558168059" Jul 14 13:56:49.075990 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:49.075952 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:56:49.206798 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:49.206764 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:56:49.305848 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:49.305804 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:56:49.406814 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:49.406774 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:56:49.811452 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:49.811415 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:49.816791 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:49.816763 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:50.003537 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:50.003507 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:50.004651 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:50.004629 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:56:52.477317 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.477281 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:56:52.577709 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.577680 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-credential-socket\") pod \"d6529521-49b4-4e6d-8b0b-959bed635a79\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " Jul 14 13:56:52.577889 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.577722 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-workload-socket\") pod \"d6529521-49b4-4e6d-8b0b-959bed635a79\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " Jul 14 13:56:52.577889 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.577747 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6529521-49b4-4e6d-8b0b-959bed635a79-istiod-ca-cert\") pod \"d6529521-49b4-4e6d-8b0b-959bed635a79\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " Jul 14 13:56:52.577889 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.577775 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shc45\" (UniqueName: \"kubernetes.io/projected/d6529521-49b4-4e6d-8b0b-959bed635a79-kube-api-access-shc45\") pod \"d6529521-49b4-4e6d-8b0b-959bed635a79\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " Jul 14 13:56:52.577889 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.577806 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-data\") pod \"d6529521-49b4-4e6d-8b0b-959bed635a79\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " Jul 14 13:56:52.577889 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.577869 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-podinfo\") pod \"d6529521-49b4-4e6d-8b0b-959bed635a79\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " Jul 14 13:56:52.578166 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.577905 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-envoy\") pod \"d6529521-49b4-4e6d-8b0b-959bed635a79\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " Jul 14 13:56:52.578166 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.577934 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-token\") pod \"d6529521-49b4-4e6d-8b0b-959bed635a79\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " Jul 14 13:56:52.578166 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.577969 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-workload-certs\") pod \"d6529521-49b4-4e6d-8b0b-959bed635a79\" (UID: \"d6529521-49b4-4e6d-8b0b-959bed635a79\") " Jul 14 13:56:52.578166 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.578012 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d6529521-49b4-4e6d-8b0b-959bed635a79" (UID: "d6529521-49b4-4e6d-8b0b-959bed635a79"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:56:52.578166 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.578026 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d6529521-49b4-4e6d-8b0b-959bed635a79" (UID: "d6529521-49b4-4e6d-8b0b-959bed635a79"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:56:52.578420 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.578245 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6529521-49b4-4e6d-8b0b-959bed635a79-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d6529521-49b4-4e6d-8b0b-959bed635a79" (UID: "d6529521-49b4-4e6d-8b0b-959bed635a79"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:56:52.578420 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.578287 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-credential-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:56:52.578420 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.578306 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-workload-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:56:52.578569 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.578504 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d6529521-49b4-4e6d-8b0b-959bed635a79" (UID: "d6529521-49b4-4e6d-8b0b-959bed635a79"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:56:52.578730 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.578704 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-data" (OuterVolumeSpecName: "istio-data") pod "d6529521-49b4-4e6d-8b0b-959bed635a79" (UID: "d6529521-49b4-4e6d-8b0b-959bed635a79"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:56:52.580446 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.580409 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d6529521-49b4-4e6d-8b0b-959bed635a79" (UID: "d6529521-49b4-4e6d-8b0b-959bed635a79"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 13:56:52.580569 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.580485 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d6529521-49b4-4e6d-8b0b-959bed635a79" (UID: "d6529521-49b4-4e6d-8b0b-959bed635a79"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:56:52.580634 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.580592 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-token" (OuterVolumeSpecName: "istio-token") pod "d6529521-49b4-4e6d-8b0b-959bed635a79" (UID: "d6529521-49b4-4e6d-8b0b-959bed635a79"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:56:52.580634 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.580610 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6529521-49b4-4e6d-8b0b-959bed635a79-kube-api-access-shc45" (OuterVolumeSpecName: "kube-api-access-shc45") pod "d6529521-49b4-4e6d-8b0b-959bed635a79" (UID: "d6529521-49b4-4e6d-8b0b-959bed635a79"). InnerVolumeSpecName "kube-api-access-shc45". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:56:52.679196 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.679159 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6529521-49b4-4e6d-8b0b-959bed635a79-istiod-ca-cert\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:56:52.679196 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.679191 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shc45\" (UniqueName: \"kubernetes.io/projected/d6529521-49b4-4e6d-8b0b-959bed635a79-kube-api-access-shc45\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:56:52.679196 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.679201 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-data\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:56:52.679445 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.679212 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-podinfo\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:56:52.679445 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.679220 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-envoy\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:56:52.679445 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.679227 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6529521-49b4-4e6d-8b0b-959bed635a79-istio-token\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:56:52.679445 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:52.679235 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6529521-49b4-4e6d-8b0b-959bed635a79-workload-certs\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:56:53.021020 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:53.020982 2583 generic.go:356] "Generic (PLEG): container finished" podID="d6529521-49b4-4e6d-8b0b-959bed635a79" containerID="a0991abba1ffe77a3fa42f5ec3ffdde05ef33d365800e89fa295c8b165200174" exitCode=0 Jul 14 13:56:53.021233 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:53.021038 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" Jul 14 13:56:53.021233 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:53.021071 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" event={"ID":"d6529521-49b4-4e6d-8b0b-959bed635a79","Type":"ContainerDied","Data":"a0991abba1ffe77a3fa42f5ec3ffdde05ef33d365800e89fa295c8b165200174"} Jul 14 13:56:53.021233 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:53.021156 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q" event={"ID":"d6529521-49b4-4e6d-8b0b-959bed635a79","Type":"ContainerDied","Data":"4f171d68c5a29d522749388e61ca758cc48bbc5275599a50750ca7b47634d9c2"} Jul 14 13:56:53.021233 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:53.021180 2583 scope.go:117] "RemoveContainer" containerID="a0991abba1ffe77a3fa42f5ec3ffdde05ef33d365800e89fa295c8b165200174" Jul 14 13:56:53.034628 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:53.034603 2583 scope.go:117] "RemoveContainer" containerID="9137dfadffaeb5d4d86e2994783726cc9aa48ade95c480c8bc3080682dab8464" Jul 14 13:56:53.053492 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:53.053139 2583 scope.go:117] "RemoveContainer" containerID="a0991abba1ffe77a3fa42f5ec3ffdde05ef33d365800e89fa295c8b165200174" Jul 14 13:56:53.054019 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:56:53.053732 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0991abba1ffe77a3fa42f5ec3ffdde05ef33d365800e89fa295c8b165200174\": container with ID starting with a0991abba1ffe77a3fa42f5ec3ffdde05ef33d365800e89fa295c8b165200174 not found: ID does not exist" containerID="a0991abba1ffe77a3fa42f5ec3ffdde05ef33d365800e89fa295c8b165200174" Jul 14 13:56:53.054019 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:53.053775 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0991abba1ffe77a3fa42f5ec3ffdde05ef33d365800e89fa295c8b165200174"} err="failed to get container status \"a0991abba1ffe77a3fa42f5ec3ffdde05ef33d365800e89fa295c8b165200174\": rpc error: code = NotFound desc = could not find container \"a0991abba1ffe77a3fa42f5ec3ffdde05ef33d365800e89fa295c8b165200174\": container with ID starting with a0991abba1ffe77a3fa42f5ec3ffdde05ef33d365800e89fa295c8b165200174 not found: ID does not exist" Jul 14 13:56:53.054019 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:53.053800 2583 scope.go:117] "RemoveContainer" containerID="9137dfadffaeb5d4d86e2994783726cc9aa48ade95c480c8bc3080682dab8464" Jul 14 13:56:53.054422 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:56:53.054338 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9137dfadffaeb5d4d86e2994783726cc9aa48ade95c480c8bc3080682dab8464\": container with ID starting with 9137dfadffaeb5d4d86e2994783726cc9aa48ade95c480c8bc3080682dab8464 not found: ID does not exist" containerID="9137dfadffaeb5d4d86e2994783726cc9aa48ade95c480c8bc3080682dab8464" Jul 14 13:56:53.054422 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:53.054370 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9137dfadffaeb5d4d86e2994783726cc9aa48ade95c480c8bc3080682dab8464"} err="failed to get container status \"9137dfadffaeb5d4d86e2994783726cc9aa48ade95c480c8bc3080682dab8464\": rpc error: code = NotFound desc = could not find container \"9137dfadffaeb5d4d86e2994783726cc9aa48ade95c480c8bc3080682dab8464\": container with ID starting with 9137dfadffaeb5d4d86e2994783726cc9aa48ade95c480c8bc3080682dab8464 not found: ID does not exist" Jul 14 13:56:53.060856 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:53.060823 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q"] Jul 14 13:56:53.064434 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:53.064408 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b3b6f94a-openshift-default-54499f77b5-szd6q"] Jul 14 13:56:54.494337 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:56:54.494301 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6529521-49b4-4e6d-8b0b-959bed635a79" path="/var/lib/kubelet/pods/d6529521-49b4-4e6d-8b0b-959bed635a79/volumes" Jul 14 13:57:00.067948 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:00.067911 2583 generic.go:356] "Generic (PLEG): container finished" podID="a85bdc03-fd2a-442f-9388-881cc2448b56" containerID="a1c715f302bb7af4099c173f4a8e84ed37be91eb7f078dd4887b47afe44aa273" exitCode=0 Jul 14 13:57:00.068457 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:00.067984 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" event={"ID":"a85bdc03-fd2a-442f-9388-881cc2448b56","Type":"ContainerDied","Data":"a1c715f302bb7af4099c173f4a8e84ed37be91eb7f078dd4887b47afe44aa273"} Jul 14 13:57:00.068457 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:00.068431 2583 scope.go:117] "RemoveContainer" containerID="a1c715f302bb7af4099c173f4a8e84ed37be91eb7f078dd4887b47afe44aa273" Jul 14 13:57:00.069454 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:00.069430 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:57:00.069516 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:00.069497 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:57:00.069558 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:00.069529 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:57:01.075412 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:01.075376 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" event={"ID":"a85bdc03-fd2a-442f-9388-881cc2448b56","Type":"ContainerStarted","Data":"be8d0dae25daa1d6a353ab27b7a2d370a08a8be23824d445999610a8fa44d24d"} Jul 14 13:57:01.811946 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:01.811903 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:57:01.813255 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:01.813225 2583 patch_prober.go:28] interesting pod/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 14 13:57:01.813388 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:01.813287 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" podUID="a85bdc03-fd2a-442f-9388-881cc2448b56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 14 13:57:02.812373 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:02.812337 2583 patch_prober.go:28] interesting pod/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 14 13:57:02.812838 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:02.812420 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" podUID="a85bdc03-fd2a-442f-9388-881cc2448b56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 14 13:57:03.812021 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:03.811983 2583 patch_prober.go:28] interesting pod/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 14 13:57:03.812226 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:03.812042 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" podUID="a85bdc03-fd2a-442f-9388-881cc2448b56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 14 13:57:04.061118 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:04.061043 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:57:04.120224 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:04.120190 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:57:04.811774 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:04.811738 2583 patch_prober.go:28] interesting pod/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 14 13:57:04.811951 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:04.811798 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" podUID="a85bdc03-fd2a-442f-9388-881cc2448b56" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 14 13:57:05.003425 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:05.003390 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:57:05.815228 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:05.815197 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:57:05.816118 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:05.816074 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:57:07.741742 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:07.741701 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:57:18.491035 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:18.491007 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 13:57:18.493266 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:57:18.493243 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 13:58:03.564626 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:03.564574 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:58:08.343379 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:08.343321 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:58:14.422699 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:14.422661 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:58:14.485468 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:14.485435 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:58:18.397972 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:18.397937 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj"] Jul 14 13:58:18.398464 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:18.398252 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" podUID="a85bdc03-fd2a-442f-9388-881cc2448b56" containerName="istio-proxy" containerID="cri-o://be8d0dae25daa1d6a353ab27b7a2d370a08a8be23824d445999610a8fa44d24d" gracePeriod=30 Jul 14 13:58:18.418115 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:18.418063 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:58:19.048695 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:19.048660 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:58:19.145351 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:19.145301 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:58:19.246103 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:19.246054 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:58:20.658349 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:20.658318 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:58:20.818591 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:20.818550 2583 patch_prober.go:28] interesting pod/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 13:58:20.818751 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:20.818615 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" podUID="a85bdc03-fd2a-442f-9388-881cc2448b56" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 13:58:23.657269 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.657239 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:58:23.735674 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.735640 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-credential-socket\") pod \"a85bdc03-fd2a-442f-9388-881cc2448b56\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " Jul 14 13:58:23.735855 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.735694 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-workload-certs\") pod \"a85bdc03-fd2a-442f-9388-881cc2448b56\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " Jul 14 13:58:23.735855 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.735735 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a85bdc03-fd2a-442f-9388-881cc2448b56-istiod-ca-cert\") pod \"a85bdc03-fd2a-442f-9388-881cc2448b56\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " Jul 14 13:58:23.735855 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.735802 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-data\") pod \"a85bdc03-fd2a-442f-9388-881cc2448b56\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " Jul 14 13:58:23.735855 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.735832 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-podinfo\") pod \"a85bdc03-fd2a-442f-9388-881cc2448b56\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " Jul 14 13:58:23.736107 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.735869 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-workload-socket\") pod \"a85bdc03-fd2a-442f-9388-881cc2448b56\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " Jul 14 13:58:23.736107 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.735893 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a85bdc03-fd2a-442f-9388-881cc2448b56" (UID: "a85bdc03-fd2a-442f-9388-881cc2448b56"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:58:23.736107 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.735906 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkwg7\" (UniqueName: \"kubernetes.io/projected/a85bdc03-fd2a-442f-9388-881cc2448b56-kube-api-access-tkwg7\") pod \"a85bdc03-fd2a-442f-9388-881cc2448b56\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " Jul 14 13:58:23.736107 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.735964 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-token\") pod \"a85bdc03-fd2a-442f-9388-881cc2448b56\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " Jul 14 13:58:23.736107 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.735992 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-envoy\") pod \"a85bdc03-fd2a-442f-9388-881cc2448b56\" (UID: \"a85bdc03-fd2a-442f-9388-881cc2448b56\") " Jul 14 13:58:23.736360 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.736118 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a85bdc03-fd2a-442f-9388-881cc2448b56" (UID: "a85bdc03-fd2a-442f-9388-881cc2448b56"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:58:23.736360 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.736223 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a85bdc03-fd2a-442f-9388-881cc2448b56-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a85bdc03-fd2a-442f-9388-881cc2448b56" (UID: "a85bdc03-fd2a-442f-9388-881cc2448b56"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:58:23.736360 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.736275 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-credential-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:58:23.736360 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.736293 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-workload-certs\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:58:23.736512 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.736488 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a85bdc03-fd2a-442f-9388-881cc2448b56" (UID: "a85bdc03-fd2a-442f-9388-881cc2448b56"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:58:23.737029 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.737001 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-data" (OuterVolumeSpecName: "istio-data") pod "a85bdc03-fd2a-442f-9388-881cc2448b56" (UID: "a85bdc03-fd2a-442f-9388-881cc2448b56"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:58:23.738472 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.738447 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a85bdc03-fd2a-442f-9388-881cc2448b56" (UID: "a85bdc03-fd2a-442f-9388-881cc2448b56"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:58:23.738570 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.738512 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a85bdc03-fd2a-442f-9388-881cc2448b56-kube-api-access-tkwg7" (OuterVolumeSpecName: "kube-api-access-tkwg7") pod "a85bdc03-fd2a-442f-9388-881cc2448b56" (UID: "a85bdc03-fd2a-442f-9388-881cc2448b56"). InnerVolumeSpecName "kube-api-access-tkwg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:58:23.738570 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.738508 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a85bdc03-fd2a-442f-9388-881cc2448b56" (UID: "a85bdc03-fd2a-442f-9388-881cc2448b56"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 13:58:23.738570 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.738530 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-token" (OuterVolumeSpecName: "istio-token") pod "a85bdc03-fd2a-442f-9388-881cc2448b56" (UID: "a85bdc03-fd2a-442f-9388-881cc2448b56"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:58:23.837425 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.837383 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkwg7\" (UniqueName: \"kubernetes.io/projected/a85bdc03-fd2a-442f-9388-881cc2448b56-kube-api-access-tkwg7\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:58:23.837425 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.837424 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-token\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:58:23.837624 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.837437 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-envoy\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:58:23.837624 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.837449 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a85bdc03-fd2a-442f-9388-881cc2448b56-istiod-ca-cert\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:58:23.837624 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.837461 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-data\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:58:23.837624 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.837474 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a85bdc03-fd2a-442f-9388-881cc2448b56-istio-podinfo\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:58:23.837624 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:23.837485 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a85bdc03-fd2a-442f-9388-881cc2448b56-workload-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 13:58:24.482380 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:24.482335 2583 generic.go:356] "Generic (PLEG): container finished" podID="a85bdc03-fd2a-442f-9388-881cc2448b56" containerID="be8d0dae25daa1d6a353ab27b7a2d370a08a8be23824d445999610a8fa44d24d" exitCode=0 Jul 14 13:58:24.482526 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:24.482385 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" event={"ID":"a85bdc03-fd2a-442f-9388-881cc2448b56","Type":"ContainerDied","Data":"be8d0dae25daa1d6a353ab27b7a2d370a08a8be23824d445999610a8fa44d24d"} Jul 14 13:58:24.482526 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:24.482397 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" Jul 14 13:58:24.482526 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:24.482431 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj" event={"ID":"a85bdc03-fd2a-442f-9388-881cc2448b56","Type":"ContainerDied","Data":"4f241c235f2298186dca3d92046eb934fe283263a5eebe07ac5f3a996dba3cd2"} Jul 14 13:58:24.482526 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:24.482453 2583 scope.go:117] "RemoveContainer" containerID="be8d0dae25daa1d6a353ab27b7a2d370a08a8be23824d445999610a8fa44d24d" Jul 14 13:58:24.492746 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:24.492718 2583 scope.go:117] "RemoveContainer" containerID="a1c715f302bb7af4099c173f4a8e84ed37be91eb7f078dd4887b47afe44aa273" Jul 14 13:58:24.500892 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:24.500873 2583 scope.go:117] "RemoveContainer" containerID="be8d0dae25daa1d6a353ab27b7a2d370a08a8be23824d445999610a8fa44d24d" Jul 14 13:58:24.501117 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:58:24.501100 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be8d0dae25daa1d6a353ab27b7a2d370a08a8be23824d445999610a8fa44d24d\": container with ID starting with be8d0dae25daa1d6a353ab27b7a2d370a08a8be23824d445999610a8fa44d24d not found: ID does not exist" containerID="be8d0dae25daa1d6a353ab27b7a2d370a08a8be23824d445999610a8fa44d24d" Jul 14 13:58:24.501172 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:24.501125 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be8d0dae25daa1d6a353ab27b7a2d370a08a8be23824d445999610a8fa44d24d"} err="failed to get container status \"be8d0dae25daa1d6a353ab27b7a2d370a08a8be23824d445999610a8fa44d24d\": rpc error: code = NotFound desc = could not find container \"be8d0dae25daa1d6a353ab27b7a2d370a08a8be23824d445999610a8fa44d24d\": container with ID starting with be8d0dae25daa1d6a353ab27b7a2d370a08a8be23824d445999610a8fa44d24d not found: ID does not exist" Jul 14 13:58:24.501172 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:24.501140 2583 scope.go:117] "RemoveContainer" containerID="a1c715f302bb7af4099c173f4a8e84ed37be91eb7f078dd4887b47afe44aa273" Jul 14 13:58:24.501390 ip-10-0-142-94 kubenswrapper[2583]: E0714 13:58:24.501372 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1c715f302bb7af4099c173f4a8e84ed37be91eb7f078dd4887b47afe44aa273\": container with ID starting with a1c715f302bb7af4099c173f4a8e84ed37be91eb7f078dd4887b47afe44aa273 not found: ID does not exist" containerID="a1c715f302bb7af4099c173f4a8e84ed37be91eb7f078dd4887b47afe44aa273" Jul 14 13:58:24.501440 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:24.501392 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1c715f302bb7af4099c173f4a8e84ed37be91eb7f078dd4887b47afe44aa273"} err="failed to get container status \"a1c715f302bb7af4099c173f4a8e84ed37be91eb7f078dd4887b47afe44aa273\": rpc error: code = NotFound desc = could not find container \"a1c715f302bb7af4099c173f4a8e84ed37be91eb7f078dd4887b47afe44aa273\": container with ID starting with a1c715f302bb7af4099c173f4a8e84ed37be91eb7f078dd4887b47afe44aa273 not found: ID does not exist" Jul 14 13:58:24.506230 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:24.506202 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj"] Jul 14 13:58:24.512913 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:24.512895 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-fe2b25e2-openshift-default-5f5f47cc79-tj9pj"] Jul 14 13:58:25.094279 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:25.094249 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:58:26.493987 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:26.493953 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a85bdc03-fd2a-442f-9388-881cc2448b56" path="/var/lib/kubelet/pods/a85bdc03-fd2a-442f-9388-881cc2448b56/volumes" Jul 14 13:58:26.978795 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:26.978761 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:58:27.048735 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:27.048698 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:58:27.197551 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:27.197516 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:58:27.293323 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:27.293282 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:58:27.844452 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:27.844411 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:58:53.701939 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:58:53.701899 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:00.127459 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:00.127418 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:00.233816 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:00.233781 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:00.336835 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:00.336799 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:16.236315 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:16.236281 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:22.267305 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:22.267266 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:22.345477 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:22.345441 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:22.417437 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:22.417407 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:22.526565 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:22.526468 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:33.058458 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:33.058416 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:33.876486 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:33.876451 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:40.286545 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:40.286504 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:40.388553 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:40.388515 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:40.489049 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:40.489019 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:40.587676 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:40.587588 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:41.347275 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:41.347223 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:41.446834 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:41.446803 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:43.122437 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:43.122399 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:49.494954 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:49.494920 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:49.578816 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:49.578780 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:50.160361 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:50.160328 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:56.395202 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:56.395165 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:56.454767 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:56.454733 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:56.640349 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:56.640314 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:56.964682 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:56.964647 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw"] Jul 14 13:59:56.965040 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:56.965027 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a85bdc03-fd2a-442f-9388-881cc2448b56" containerName="istio-proxy" Jul 14 13:59:56.965120 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:56.965041 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a85bdc03-fd2a-442f-9388-881cc2448b56" containerName="istio-proxy" Jul 14 13:59:56.965120 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:56.965052 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a85bdc03-fd2a-442f-9388-881cc2448b56" containerName="istio-proxy" Jul 14 13:59:56.965120 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:56.965060 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a85bdc03-fd2a-442f-9388-881cc2448b56" containerName="istio-proxy" Jul 14 13:59:56.965120 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:56.965094 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6529521-49b4-4e6d-8b0b-959bed635a79" containerName="istio-proxy" Jul 14 13:59:56.965120 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:56.965101 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6529521-49b4-4e6d-8b0b-959bed635a79" containerName="istio-proxy" Jul 14 13:59:56.965120 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:56.965113 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6529521-49b4-4e6d-8b0b-959bed635a79" containerName="istio-proxy" Jul 14 13:59:56.965120 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:56.965118 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6529521-49b4-4e6d-8b0b-959bed635a79" containerName="istio-proxy" Jul 14 13:59:56.965486 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:56.965181 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6529521-49b4-4e6d-8b0b-959bed635a79" containerName="istio-proxy" Jul 14 13:59:56.965486 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:56.965188 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a85bdc03-fd2a-442f-9388-881cc2448b56" containerName="istio-proxy" Jul 14 13:59:56.965486 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:56.965194 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a85bdc03-fd2a-442f-9388-881cc2448b56" containerName="istio-proxy" Jul 14 13:59:56.965486 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:56.965200 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6529521-49b4-4e6d-8b0b-959bed635a79" containerName="istio-proxy" Jul 14 13:59:56.968272 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:56.968254 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:56.971835 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:56.971813 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-50aeb779-openshift-default-dockercfg-9kb8t\"" Jul 14 13:59:57.006143 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.006115 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:57.007057 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.007036 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw"] Jul 14 13:59:57.095900 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.095865 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/857f8494-cef7-456c-bf27-8854891e1f1a-istiod-ca-cert\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.096071 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.095910 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-credential-socket\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.096071 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.095991 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76jbd\" (UniqueName: \"kubernetes.io/projected/857f8494-cef7-456c-bf27-8854891e1f1a-kube-api-access-76jbd\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.096071 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.096036 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-workload-socket\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.096071 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.096054 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-istio-data\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.096325 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.096103 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/857f8494-cef7-456c-bf27-8854891e1f1a-istio-podinfo\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.096325 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.096195 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-workload-certs\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.096325 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.096237 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-istio-envoy\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.096325 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.096291 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/857f8494-cef7-456c-bf27-8854891e1f1a-istio-token\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.106407 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.106376 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:57.197733 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.197694 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/857f8494-cef7-456c-bf27-8854891e1f1a-istiod-ca-cert\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.197733 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.197738 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-credential-socket\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.197997 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.197864 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76jbd\" (UniqueName: \"kubernetes.io/projected/857f8494-cef7-456c-bf27-8854891e1f1a-kube-api-access-76jbd\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.197997 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.197928 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-workload-socket\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.197997 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.197954 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-istio-data\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.198203 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.197997 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/857f8494-cef7-456c-bf27-8854891e1f1a-istio-podinfo\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.198203 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.198073 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-workload-certs\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.198203 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.198111 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-credential-socket\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.198203 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.198126 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-istio-envoy\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.198203 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.198199 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/857f8494-cef7-456c-bf27-8854891e1f1a-istio-token\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.198465 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.198368 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-istio-data\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.198465 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.198394 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-workload-socket\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.198574 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.198469 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-workload-certs\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.198574 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.198537 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/857f8494-cef7-456c-bf27-8854891e1f1a-istiod-ca-cert\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.200334 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.200314 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-istio-envoy\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.200632 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.200614 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/857f8494-cef7-456c-bf27-8854891e1f1a-istio-podinfo\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.207119 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.207070 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/857f8494-cef7-456c-bf27-8854891e1f1a-istio-token\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.207294 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.207278 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76jbd\" (UniqueName: \"kubernetes.io/projected/857f8494-cef7-456c-bf27-8854891e1f1a-kube-api-access-76jbd\") pod \"e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.278661 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.278630 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:57.421164 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.421139 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw"] Jul 14 13:59:57.429733 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.429684 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:59:57.429857 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.429770 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:59:57.429857 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.429815 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 13:59:57.921186 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.921153 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" event={"ID":"857f8494-cef7-456c-bf27-8854891e1f1a","Type":"ContainerStarted","Data":"842751b9e1f19c575a554db5a98133d20bcea69de667a1801264ddafa12dc877"} Jul 14 13:59:57.921394 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.921212 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" event={"ID":"857f8494-cef7-456c-bf27-8854891e1f1a","Type":"ContainerStarted","Data":"9d585b82bcf2a985b633090bff5d95583366fb062838e05c31656ea8b574765d"} Jul 14 13:59:57.946740 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:57.946691 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" podStartSLOduration=1.946672993 podStartE2EDuration="1.946672993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:59:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:59:57.944631619 +0000 UTC m=+1961.471782256" watchObservedRunningTime="2026-07-14 13:59:57.946672993 +0000 UTC m=+1961.473823629" Jul 14 13:59:58.278907 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:58.278869 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:58.283614 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:58.283583 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:58.927827 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:58.927800 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:58.928835 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:58.928813 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 13:59:59.877799 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:59.877760 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 13:59:59.973209 ip-10-0-142-94 kubenswrapper[2583]: I0714 13:59:59.973180 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:00.519497 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:00.519454 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:00.730426 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:00.730373 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:00.844793 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:00.844706 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:04.395593 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:04.395556 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:05.100963 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:05.100928 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:05.252353 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:05.252317 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:06.261726 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:06.261691 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:12.715275 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:12.715238 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:12.828497 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:12.828458 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:13.929697 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:13.929667 2583 patch_prober.go:28] interesting pod/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 14 14:00:13.930050 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:13.929732 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" podUID="857f8494-cef7-456c-bf27-8854891e1f1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 14 14:00:14.005154 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:14.005121 2583 generic.go:356] "Generic (PLEG): container finished" podID="857f8494-cef7-456c-bf27-8854891e1f1a" containerID="842751b9e1f19c575a554db5a98133d20bcea69de667a1801264ddafa12dc877" exitCode=0 Jul 14 14:00:14.005338 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:14.005203 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" event={"ID":"857f8494-cef7-456c-bf27-8854891e1f1a","Type":"ContainerDied","Data":"842751b9e1f19c575a554db5a98133d20bcea69de667a1801264ddafa12dc877"} Jul 14 14:00:14.005676 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:14.005659 2583 scope.go:117] "RemoveContainer" containerID="842751b9e1f19c575a554db5a98133d20bcea69de667a1801264ddafa12dc877" Jul 14 14:00:14.006734 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:14.006698 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:00:14.006827 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:14.006801 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:00:14.006871 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:14.006840 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:00:15.012352 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:15.012317 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" event={"ID":"857f8494-cef7-456c-bf27-8854891e1f1a","Type":"ContainerStarted","Data":"7052619cf6b44f3736d86248e3fbf45c467c932f3ebf118b54682537b51ea2f7"} Jul 14 14:00:15.279921 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:15.279829 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 14:00:15.280103 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:15.280032 2583 patch_prober.go:28] interesting pod/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 14 14:00:15.280165 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:15.280074 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" podUID="857f8494-cef7-456c-bf27-8854891e1f1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 14 14:00:16.279138 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:16.279065 2583 patch_prober.go:28] interesting pod/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 14 14:00:16.279562 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:16.279155 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" podUID="857f8494-cef7-456c-bf27-8854891e1f1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 14 14:00:17.279385 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:17.279346 2583 patch_prober.go:28] interesting pod/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 14 14:00:17.279798 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:17.279402 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" podUID="857f8494-cef7-456c-bf27-8854891e1f1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 14 14:00:18.279653 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:18.279618 2583 patch_prober.go:28] interesting pod/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 14 14:00:18.280030 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:18.279677 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" podUID="857f8494-cef7-456c-bf27-8854891e1f1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 14 14:00:19.279577 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:19.279544 2583 patch_prober.go:28] interesting pod/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 14 14:00:19.279753 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:19.279600 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" podUID="857f8494-cef7-456c-bf27-8854891e1f1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 14 14:00:20.283929 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:20.283894 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 14:00:20.284405 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:20.284384 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 14:00:20.285185 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:20.285165 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 14:00:22.843973 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:22.843934 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:27.394507 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:27.394472 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:33.696782 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:33.696745 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:33.786518 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:33.786485 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:38.093909 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:38.093871 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw"] Jul 14 14:00:38.094443 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:38.094175 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" podUID="857f8494-cef7-456c-bf27-8854891e1f1a" containerName="istio-proxy" containerID="cri-o://7052619cf6b44f3736d86248e3fbf45c467c932f3ebf118b54682537b51ea2f7" gracePeriod=30 Jul 14 14:00:38.115998 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:38.115960 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:38.344631 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:38.344540 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:38.513132 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:38.513092 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:38.610286 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:38.610199 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:39.594250 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:39.594210 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:42.280281 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:42.280248 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:42.381150 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:42.381118 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:43.345449 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.345424 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 14:00:43.437909 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.437812 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-istio-envoy\") pod \"857f8494-cef7-456c-bf27-8854891e1f1a\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " Jul 14 14:00:43.437909 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.437857 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-workload-socket\") pod \"857f8494-cef7-456c-bf27-8854891e1f1a\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " Jul 14 14:00:43.437909 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.437887 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-workload-certs\") pod \"857f8494-cef7-456c-bf27-8854891e1f1a\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " Jul 14 14:00:43.438232 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.437928 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/857f8494-cef7-456c-bf27-8854891e1f1a-istio-token\") pod \"857f8494-cef7-456c-bf27-8854891e1f1a\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " Jul 14 14:00:43.438232 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.437968 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/857f8494-cef7-456c-bf27-8854891e1f1a-istio-podinfo\") pod \"857f8494-cef7-456c-bf27-8854891e1f1a\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " Jul 14 14:00:43.438232 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.438013 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-credential-socket\") pod \"857f8494-cef7-456c-bf27-8854891e1f1a\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " Jul 14 14:00:43.438232 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.438038 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-istio-data\") pod \"857f8494-cef7-456c-bf27-8854891e1f1a\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " Jul 14 14:00:43.438232 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.438056 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76jbd\" (UniqueName: \"kubernetes.io/projected/857f8494-cef7-456c-bf27-8854891e1f1a-kube-api-access-76jbd\") pod \"857f8494-cef7-456c-bf27-8854891e1f1a\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " Jul 14 14:00:43.438232 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.438115 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/857f8494-cef7-456c-bf27-8854891e1f1a-istiod-ca-cert\") pod \"857f8494-cef7-456c-bf27-8854891e1f1a\" (UID: \"857f8494-cef7-456c-bf27-8854891e1f1a\") " Jul 14 14:00:43.438539 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.438264 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "857f8494-cef7-456c-bf27-8854891e1f1a" (UID: "857f8494-cef7-456c-bf27-8854891e1f1a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:00:43.438539 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.438334 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "857f8494-cef7-456c-bf27-8854891e1f1a" (UID: "857f8494-cef7-456c-bf27-8854891e1f1a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:00:43.438539 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.438431 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "857f8494-cef7-456c-bf27-8854891e1f1a" (UID: "857f8494-cef7-456c-bf27-8854891e1f1a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:00:43.438539 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.438461 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-workload-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:00:43.438539 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.438479 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-workload-certs\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:00:43.438768 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.438735 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/857f8494-cef7-456c-bf27-8854891e1f1a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "857f8494-cef7-456c-bf27-8854891e1f1a" (UID: "857f8494-cef7-456c-bf27-8854891e1f1a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:00:43.439215 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.439183 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-istio-data" (OuterVolumeSpecName: "istio-data") pod "857f8494-cef7-456c-bf27-8854891e1f1a" (UID: "857f8494-cef7-456c-bf27-8854891e1f1a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:00:43.440433 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.440405 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/857f8494-cef7-456c-bf27-8854891e1f1a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "857f8494-cef7-456c-bf27-8854891e1f1a" (UID: "857f8494-cef7-456c-bf27-8854891e1f1a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 14:00:43.440552 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.440530 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/857f8494-cef7-456c-bf27-8854891e1f1a-kube-api-access-76jbd" (OuterVolumeSpecName: "kube-api-access-76jbd") pod "857f8494-cef7-456c-bf27-8854891e1f1a" (UID: "857f8494-cef7-456c-bf27-8854891e1f1a"). InnerVolumeSpecName "kube-api-access-76jbd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:00:43.440618 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.440560 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "857f8494-cef7-456c-bf27-8854891e1f1a" (UID: "857f8494-cef7-456c-bf27-8854891e1f1a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:00:43.440618 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.440564 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/857f8494-cef7-456c-bf27-8854891e1f1a-istio-token" (OuterVolumeSpecName: "istio-token") pod "857f8494-cef7-456c-bf27-8854891e1f1a" (UID: "857f8494-cef7-456c-bf27-8854891e1f1a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:00:43.539261 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.539229 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/857f8494-cef7-456c-bf27-8854891e1f1a-istio-podinfo\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:00:43.539261 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.539256 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-credential-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:00:43.539261 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.539267 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-istio-data\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:00:43.539494 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.539275 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76jbd\" (UniqueName: \"kubernetes.io/projected/857f8494-cef7-456c-bf27-8854891e1f1a-kube-api-access-76jbd\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:00:43.539494 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.539283 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/857f8494-cef7-456c-bf27-8854891e1f1a-istiod-ca-cert\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:00:43.539494 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.539292 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/857f8494-cef7-456c-bf27-8854891e1f1a-istio-envoy\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:00:43.539494 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:43.539299 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/857f8494-cef7-456c-bf27-8854891e1f1a-istio-token\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:00:44.171890 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:44.171854 2583 generic.go:356] "Generic (PLEG): container finished" podID="857f8494-cef7-456c-bf27-8854891e1f1a" containerID="7052619cf6b44f3736d86248e3fbf45c467c932f3ebf118b54682537b51ea2f7" exitCode=0 Jul 14 14:00:44.172067 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:44.171898 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" Jul 14 14:00:44.172067 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:44.171941 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" event={"ID":"857f8494-cef7-456c-bf27-8854891e1f1a","Type":"ContainerDied","Data":"7052619cf6b44f3736d86248e3fbf45c467c932f3ebf118b54682537b51ea2f7"} Jul 14 14:00:44.172067 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:44.171999 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw" event={"ID":"857f8494-cef7-456c-bf27-8854891e1f1a","Type":"ContainerDied","Data":"9d585b82bcf2a985b633090bff5d95583366fb062838e05c31656ea8b574765d"} Jul 14 14:00:44.172067 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:44.172027 2583 scope.go:117] "RemoveContainer" containerID="7052619cf6b44f3736d86248e3fbf45c467c932f3ebf118b54682537b51ea2f7" Jul 14 14:00:44.181074 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:44.181054 2583 scope.go:117] "RemoveContainer" containerID="842751b9e1f19c575a554db5a98133d20bcea69de667a1801264ddafa12dc877" Jul 14 14:00:44.189937 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:44.189917 2583 scope.go:117] "RemoveContainer" containerID="7052619cf6b44f3736d86248e3fbf45c467c932f3ebf118b54682537b51ea2f7" Jul 14 14:00:44.190268 ip-10-0-142-94 kubenswrapper[2583]: E0714 14:00:44.190243 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7052619cf6b44f3736d86248e3fbf45c467c932f3ebf118b54682537b51ea2f7\": container with ID starting with 7052619cf6b44f3736d86248e3fbf45c467c932f3ebf118b54682537b51ea2f7 not found: ID does not exist" containerID="7052619cf6b44f3736d86248e3fbf45c467c932f3ebf118b54682537b51ea2f7" Jul 14 14:00:44.190329 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:44.190279 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7052619cf6b44f3736d86248e3fbf45c467c932f3ebf118b54682537b51ea2f7"} err="failed to get container status \"7052619cf6b44f3736d86248e3fbf45c467c932f3ebf118b54682537b51ea2f7\": rpc error: code = NotFound desc = could not find container \"7052619cf6b44f3736d86248e3fbf45c467c932f3ebf118b54682537b51ea2f7\": container with ID starting with 7052619cf6b44f3736d86248e3fbf45c467c932f3ebf118b54682537b51ea2f7 not found: ID does not exist" Jul 14 14:00:44.190329 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:44.190298 2583 scope.go:117] "RemoveContainer" containerID="842751b9e1f19c575a554db5a98133d20bcea69de667a1801264ddafa12dc877" Jul 14 14:00:44.190578 ip-10-0-142-94 kubenswrapper[2583]: E0714 14:00:44.190559 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"842751b9e1f19c575a554db5a98133d20bcea69de667a1801264ddafa12dc877\": container with ID starting with 842751b9e1f19c575a554db5a98133d20bcea69de667a1801264ddafa12dc877 not found: ID does not exist" containerID="842751b9e1f19c575a554db5a98133d20bcea69de667a1801264ddafa12dc877" Jul 14 14:00:44.190628 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:44.190585 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"842751b9e1f19c575a554db5a98133d20bcea69de667a1801264ddafa12dc877"} err="failed to get container status \"842751b9e1f19c575a554db5a98133d20bcea69de667a1801264ddafa12dc877\": rpc error: code = NotFound desc = could not find container \"842751b9e1f19c575a554db5a98133d20bcea69de667a1801264ddafa12dc877\": container with ID starting with 842751b9e1f19c575a554db5a98133d20bcea69de667a1801264ddafa12dc877 not found: ID does not exist" Jul 14 14:00:44.196833 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:44.196810 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw"] Jul 14 14:00:44.201489 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:44.201466 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-50aeb779-openshift-default-5dd956f7dc-rhhtw"] Jul 14 14:00:44.493917 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:44.493836 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="857f8494-cef7-456c-bf27-8854891e1f1a" path="/var/lib/kubelet/pods/857f8494-cef7-456c-bf27-8854891e1f1a/volumes" Jul 14 14:00:45.474597 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:45.474563 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:45.542808 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:45.542773 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:51.957850 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:51.957815 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:52.067276 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:52.067241 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:58.823364 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:58.823324 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:59.132909 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.132817 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh"] Jul 14 14:00:59.133291 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.133274 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="857f8494-cef7-456c-bf27-8854891e1f1a" containerName="istio-proxy" Jul 14 14:00:59.133371 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.133293 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="857f8494-cef7-456c-bf27-8854891e1f1a" containerName="istio-proxy" Jul 14 14:00:59.133371 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.133305 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="857f8494-cef7-456c-bf27-8854891e1f1a" containerName="istio-proxy" Jul 14 14:00:59.133371 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.133310 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="857f8494-cef7-456c-bf27-8854891e1f1a" containerName="istio-proxy" Jul 14 14:00:59.133522 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.133389 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="857f8494-cef7-456c-bf27-8854891e1f1a" containerName="istio-proxy" Jul 14 14:00:59.133522 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.133399 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="857f8494-cef7-456c-bf27-8854891e1f1a" containerName="istio-proxy" Jul 14 14:00:59.135871 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.135828 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.139782 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.139758 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4ca3631f-openshift-default-dockercfg-ngb6c\"" Jul 14 14:00:59.156910 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.156885 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh"] Jul 14 14:00:59.285693 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.285656 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-podinfo\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.285693 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.285699 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-credential-socket\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.285902 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.285720 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-workload-socket\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.285902 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.285797 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l48pm\" (UniqueName: \"kubernetes.io/projected/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-kube-api-access-l48pm\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.285902 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.285831 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-workload-certs\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.285902 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.285865 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-envoy\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.285902 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.285888 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-token\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.286063 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.285905 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istiod-ca-cert\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.286063 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.285937 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-data\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.386483 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.386392 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-podinfo\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.386483 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.386439 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-credential-socket\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.386483 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.386461 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-workload-socket\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.386739 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.386502 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l48pm\" (UniqueName: \"kubernetes.io/projected/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-kube-api-access-l48pm\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.386739 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.386527 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-workload-certs\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.386739 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.386561 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-envoy\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.386739 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.386593 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-token\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.386739 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.386620 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istiod-ca-cert\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.386739 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.386661 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-data\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.387032 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.386928 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-workload-socket\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.387032 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.386948 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-credential-socket\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.387170 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.387060 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-workload-certs\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.387170 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.387103 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-data\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.387463 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.387445 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istiod-ca-cert\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.388785 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.388755 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-envoy\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.389059 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.389041 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-podinfo\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.396528 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.396494 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l48pm\" (UniqueName: \"kubernetes.io/projected/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-kube-api-access-l48pm\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.396656 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.396500 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-token\") pod \"e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.449427 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.449392 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:00:59.542595 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.542561 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:00:59.589723 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.589699 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh"] Jul 14 14:00:59.591743 ip-10-0-142-94 kubenswrapper[2583]: W0714 14:00:59.591713 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaf4da40_9b14_459f_856c_b1d7aeec7bf3.slice/crio-436b46bcbc94e6a25ae14e00fc644070d50d5a006896ae4bb10b619bcf79182e WatchSource:0}: Error finding container 436b46bcbc94e6a25ae14e00fc644070d50d5a006896ae4bb10b619bcf79182e: Status 404 returned error can't find the container with id 436b46bcbc94e6a25ae14e00fc644070d50d5a006896ae4bb10b619bcf79182e Jul 14 14:00:59.593956 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.593923 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:00:59.594027 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.594012 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:00:59.594064 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.594047 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:00:59.642389 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:00:59.642359 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:01:00.255031 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:00.254996 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" event={"ID":"aaf4da40-9b14-459f-856c-b1d7aeec7bf3","Type":"ContainerStarted","Data":"28714fb54e29e91fae37cf4c66b3bf9e134e57f109ba32fd37ca4cfdb62f7af0"} Jul 14 14:01:00.255440 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:00.255075 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" event={"ID":"aaf4da40-9b14-459f-856c-b1d7aeec7bf3","Type":"ContainerStarted","Data":"436b46bcbc94e6a25ae14e00fc644070d50d5a006896ae4bb10b619bcf79182e"} Jul 14 14:01:00.279012 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:00.278964 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" podStartSLOduration=1.278949753 podStartE2EDuration="1.278949753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:00:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:01:00.278097808 +0000 UTC m=+2023.805248454" watchObservedRunningTime="2026-07-14 14:01:00.278949753 +0000 UTC m=+2023.806100393" Jul 14 14:01:00.449763 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:00.449728 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:01:00.454903 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:00.454878 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:01:01.260341 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:01.260308 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:01:01.261267 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:01.261240 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:01:02.364922 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:02.364889 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:01:02.476287 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:02.476252 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:01:02.665031 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:02.664940 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:01:02.765180 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:02.765146 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:01:03.094236 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:03.094201 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:01:03.193468 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:03.193438 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:01:06.763586 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:06.763515 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:01:16.261767 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:16.261725 2583 patch_prober.go:28] interesting pod/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 14 14:01:16.262287 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:16.261803 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" podUID="aaf4da40-9b14-459f-856c-b1d7aeec7bf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 14 14:01:16.341644 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:16.341609 2583 generic.go:356] "Generic (PLEG): container finished" podID="aaf4da40-9b14-459f-856c-b1d7aeec7bf3" containerID="28714fb54e29e91fae37cf4c66b3bf9e134e57f109ba32fd37ca4cfdb62f7af0" exitCode=0 Jul 14 14:01:16.341826 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:16.341666 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" event={"ID":"aaf4da40-9b14-459f-856c-b1d7aeec7bf3","Type":"ContainerDied","Data":"28714fb54e29e91fae37cf4c66b3bf9e134e57f109ba32fd37ca4cfdb62f7af0"} Jul 14 14:01:16.342162 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:16.342127 2583 scope.go:117] "RemoveContainer" containerID="28714fb54e29e91fae37cf4c66b3bf9e134e57f109ba32fd37ca4cfdb62f7af0" Jul 14 14:01:16.343226 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:16.343191 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:01:16.343292 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:16.343280 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:01:16.343328 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:16.343316 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:01:16.496847 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:16.496810 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:01:17.352700 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:17.352653 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" event={"ID":"aaf4da40-9b14-459f-856c-b1d7aeec7bf3","Type":"ContainerStarted","Data":"e81db5ef0a13314998b6cbafa226550885fa5a15ec5a5ac7d7e64c375c0f76c0"} Jul 14 14:01:17.450047 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:17.449944 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:01:17.450298 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:17.450261 2583 patch_prober.go:28] interesting pod/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 14 14:01:17.450373 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:17.450341 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" podUID="aaf4da40-9b14-459f-856c-b1d7aeec7bf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 14 14:01:18.450320 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:18.450278 2583 patch_prober.go:28] interesting pod/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 14 14:01:18.450754 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:18.450355 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" podUID="aaf4da40-9b14-459f-856c-b1d7aeec7bf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 14 14:01:19.450130 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:19.450066 2583 patch_prober.go:28] interesting pod/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 14 14:01:19.450353 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:19.450183 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" podUID="aaf4da40-9b14-459f-856c-b1d7aeec7bf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 14 14:01:19.864778 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:19.864745 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:01:20.449818 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:20.449773 2583 patch_prober.go:28] interesting pod/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 14 14:01:20.450037 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:20.449837 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" podUID="aaf4da40-9b14-459f-856c-b1d7aeec7bf3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 14 14:01:21.454653 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:21.454619 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:01:21.455020 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:21.454896 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:01:21.455566 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:21.455544 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:01:55.608973 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:01:55.608919 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:02:00.099730 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:02:00.099693 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:02:05.171779 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:02:05.171739 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:02:18.522407 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:02:18.522379 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 14:02:18.523949 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:02:18.523928 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 14:02:20.871187 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:02:20.871152 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:02:21.013004 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:02:21.012968 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:02:41.491531 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:02:41.491490 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:02:42.515980 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:02:42.515944 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:02:48.755436 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:02:48.755372 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:02:54.950046 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:02:54.949992 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:02:55.032584 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:02:55.032534 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:02.358532 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:02.358486 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh"] Jul 14 14:03:02.358961 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:02.358768 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" podUID="aaf4da40-9b14-459f-856c-b1d7aeec7bf3" containerName="istio-proxy" containerID="cri-o://e81db5ef0a13314998b6cbafa226550885fa5a15ec5a5ac7d7e64c375c0f76c0" gracePeriod=30 Jul 14 14:03:02.380968 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:02.380936 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:02.606450 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:02.606413 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:03.653855 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:03.653818 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:03.760521 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:03.760475 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:03.857243 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:03.857206 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:04.571147 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:04.571111 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:04.680858 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:04.680826 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:04.785976 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:04.785940 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:06.457717 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:06.457674 2583 patch_prober.go:28] interesting pod/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 14:03:06.458099 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:06.457739 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" podUID="aaf4da40-9b14-459f-856c-b1d7aeec7bf3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 14:03:07.622267 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.622208 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:03:07.697867 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.697839 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-token\") pod \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " Jul 14 14:03:07.698048 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.697874 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istiod-ca-cert\") pod \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " Jul 14 14:03:07.698048 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.697904 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-credential-socket\") pod \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " Jul 14 14:03:07.698048 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.697965 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-workload-certs\") pod \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " Jul 14 14:03:07.698048 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.698008 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-data\") pod \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " Jul 14 14:03:07.698048 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.698046 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-podinfo\") pod \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " Jul 14 14:03:07.698344 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.698074 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-envoy\") pod \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " Jul 14 14:03:07.698344 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.698134 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l48pm\" (UniqueName: \"kubernetes.io/projected/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-kube-api-access-l48pm\") pod \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " Jul 14 14:03:07.698344 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.698222 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-workload-socket\") pod \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\" (UID: \"aaf4da40-9b14-459f-856c-b1d7aeec7bf3\") " Jul 14 14:03:07.698504 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.698344 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aaf4da40-9b14-459f-856c-b1d7aeec7bf3" (UID: "aaf4da40-9b14-459f-856c-b1d7aeec7bf3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:03:07.698504 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.698377 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aaf4da40-9b14-459f-856c-b1d7aeec7bf3" (UID: "aaf4da40-9b14-459f-856c-b1d7aeec7bf3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:03:07.698604 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.698566 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aaf4da40-9b14-459f-856c-b1d7aeec7bf3" (UID: "aaf4da40-9b14-459f-856c-b1d7aeec7bf3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:03:07.698604 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.698580 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-workload-certs\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:03:07.698604 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.698600 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istiod-ca-cert\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:03:07.698831 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.698785 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aaf4da40-9b14-459f-856c-b1d7aeec7bf3" (UID: "aaf4da40-9b14-459f-856c-b1d7aeec7bf3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:03:07.699022 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.698994 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-data" (OuterVolumeSpecName: "istio-data") pod "aaf4da40-9b14-459f-856c-b1d7aeec7bf3" (UID: "aaf4da40-9b14-459f-856c-b1d7aeec7bf3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:03:07.700188 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.700152 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-token" (OuterVolumeSpecName: "istio-token") pod "aaf4da40-9b14-459f-856c-b1d7aeec7bf3" (UID: "aaf4da40-9b14-459f-856c-b1d7aeec7bf3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:03:07.700794 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.700770 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aaf4da40-9b14-459f-856c-b1d7aeec7bf3" (UID: "aaf4da40-9b14-459f-856c-b1d7aeec7bf3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 14:03:07.700794 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.700787 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-kube-api-access-l48pm" (OuterVolumeSpecName: "kube-api-access-l48pm") pod "aaf4da40-9b14-459f-856c-b1d7aeec7bf3" (UID: "aaf4da40-9b14-459f-856c-b1d7aeec7bf3"). InnerVolumeSpecName "kube-api-access-l48pm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:03:07.700959 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.700882 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aaf4da40-9b14-459f-856c-b1d7aeec7bf3" (UID: "aaf4da40-9b14-459f-856c-b1d7aeec7bf3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:03:07.701255 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.701233 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:07.799983 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.799932 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l48pm\" (UniqueName: \"kubernetes.io/projected/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-kube-api-access-l48pm\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:03:07.799983 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.799976 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-workload-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:03:07.799983 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.799992 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-token\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:03:07.800250 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.800004 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-credential-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:03:07.800250 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.800016 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-data\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:03:07.800250 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.800029 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-podinfo\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:03:07.800250 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.800040 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aaf4da40-9b14-459f-856c-b1d7aeec7bf3-istio-envoy\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:03:07.901817 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.901709 2583 generic.go:356] "Generic (PLEG): container finished" podID="aaf4da40-9b14-459f-856c-b1d7aeec7bf3" containerID="e81db5ef0a13314998b6cbafa226550885fa5a15ec5a5ac7d7e64c375c0f76c0" exitCode=0 Jul 14 14:03:07.901817 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.901774 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" Jul 14 14:03:07.901817 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.901790 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" event={"ID":"aaf4da40-9b14-459f-856c-b1d7aeec7bf3","Type":"ContainerDied","Data":"e81db5ef0a13314998b6cbafa226550885fa5a15ec5a5ac7d7e64c375c0f76c0"} Jul 14 14:03:07.902207 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.901850 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh" event={"ID":"aaf4da40-9b14-459f-856c-b1d7aeec7bf3","Type":"ContainerDied","Data":"436b46bcbc94e6a25ae14e00fc644070d50d5a006896ae4bb10b619bcf79182e"} Jul 14 14:03:07.902207 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.901873 2583 scope.go:117] "RemoveContainer" containerID="e81db5ef0a13314998b6cbafa226550885fa5a15ec5a5ac7d7e64c375c0f76c0" Jul 14 14:03:07.911567 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.911550 2583 scope.go:117] "RemoveContainer" containerID="28714fb54e29e91fae37cf4c66b3bf9e134e57f109ba32fd37ca4cfdb62f7af0" Jul 14 14:03:07.920374 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.920353 2583 scope.go:117] "RemoveContainer" containerID="e81db5ef0a13314998b6cbafa226550885fa5a15ec5a5ac7d7e64c375c0f76c0" Jul 14 14:03:07.920718 ip-10-0-142-94 kubenswrapper[2583]: E0714 14:03:07.920688 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e81db5ef0a13314998b6cbafa226550885fa5a15ec5a5ac7d7e64c375c0f76c0\": container with ID starting with e81db5ef0a13314998b6cbafa226550885fa5a15ec5a5ac7d7e64c375c0f76c0 not found: ID does not exist" containerID="e81db5ef0a13314998b6cbafa226550885fa5a15ec5a5ac7d7e64c375c0f76c0" Jul 14 14:03:07.920780 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.920729 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e81db5ef0a13314998b6cbafa226550885fa5a15ec5a5ac7d7e64c375c0f76c0"} err="failed to get container status \"e81db5ef0a13314998b6cbafa226550885fa5a15ec5a5ac7d7e64c375c0f76c0\": rpc error: code = NotFound desc = could not find container \"e81db5ef0a13314998b6cbafa226550885fa5a15ec5a5ac7d7e64c375c0f76c0\": container with ID starting with e81db5ef0a13314998b6cbafa226550885fa5a15ec5a5ac7d7e64c375c0f76c0 not found: ID does not exist" Jul 14 14:03:07.920780 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.920748 2583 scope.go:117] "RemoveContainer" containerID="28714fb54e29e91fae37cf4c66b3bf9e134e57f109ba32fd37ca4cfdb62f7af0" Jul 14 14:03:07.920993 ip-10-0-142-94 kubenswrapper[2583]: E0714 14:03:07.920961 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28714fb54e29e91fae37cf4c66b3bf9e134e57f109ba32fd37ca4cfdb62f7af0\": container with ID starting with 28714fb54e29e91fae37cf4c66b3bf9e134e57f109ba32fd37ca4cfdb62f7af0 not found: ID does not exist" containerID="28714fb54e29e91fae37cf4c66b3bf9e134e57f109ba32fd37ca4cfdb62f7af0" Jul 14 14:03:07.921034 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.921008 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28714fb54e29e91fae37cf4c66b3bf9e134e57f109ba32fd37ca4cfdb62f7af0"} err="failed to get container status \"28714fb54e29e91fae37cf4c66b3bf9e134e57f109ba32fd37ca4cfdb62f7af0\": rpc error: code = NotFound desc = could not find container \"28714fb54e29e91fae37cf4c66b3bf9e134e57f109ba32fd37ca4cfdb62f7af0\": container with ID starting with 28714fb54e29e91fae37cf4c66b3bf9e134e57f109ba32fd37ca4cfdb62f7af0 not found: ID does not exist" Jul 14 14:03:07.929813 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.929778 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh"] Jul 14 14:03:07.932980 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:07.932955 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4ca3631f-openshift-default-75b7696c55-xrfbh"] Jul 14 14:03:08.493274 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:08.493200 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaf4da40-9b14-459f-856c-b1d7aeec7bf3" path="/var/lib/kubelet/pods/aaf4da40-9b14-459f-856c-b1d7aeec7bf3/volumes" Jul 14 14:03:12.591629 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:12.591588 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:12.692126 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:12.692063 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:13.266873 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:13.266835 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:19.993004 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:19.992965 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:20.098269 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:20.098233 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:20.192290 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:20.192264 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:30.757866 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:30.757834 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:31.104726 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:31.104635 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:31.530664 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:31.530623 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:52.054340 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:52.054283 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:58.556995 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:58.556928 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:58.658540 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:58.658485 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:58.964848 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:58.964743 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:59.235622 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.235529 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd"] Jul 14 14:03:59.235929 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.235917 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaf4da40-9b14-459f-856c-b1d7aeec7bf3" containerName="istio-proxy" Jul 14 14:03:59.235996 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.235930 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaf4da40-9b14-459f-856c-b1d7aeec7bf3" containerName="istio-proxy" Jul 14 14:03:59.235996 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.235939 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaf4da40-9b14-459f-856c-b1d7aeec7bf3" containerName="istio-proxy" Jul 14 14:03:59.235996 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.235945 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaf4da40-9b14-459f-856c-b1d7aeec7bf3" containerName="istio-proxy" Jul 14 14:03:59.236179 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.236014 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaf4da40-9b14-459f-856c-b1d7aeec7bf3" containerName="istio-proxy" Jul 14 14:03:59.236179 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.236022 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaf4da40-9b14-459f-856c-b1d7aeec7bf3" containerName="istio-proxy" Jul 14 14:03:59.239258 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.239238 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.241956 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.241934 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-370ee8-openshift-default-dockercfg-9hqj4\"" Jul 14 14:03:59.258591 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.258566 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd"] Jul 14 14:03:59.273879 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.273852 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:59.367138 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.367070 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-istio-envoy\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.367288 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.367225 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/086f87d9-dafc-4c6d-b660-d175a2723122-istiod-ca-cert\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.367288 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.367266 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chd7r\" (UniqueName: \"kubernetes.io/projected/086f87d9-dafc-4c6d-b660-d175a2723122-kube-api-access-chd7r\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.367383 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.367349 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/086f87d9-dafc-4c6d-b660-d175a2723122-istio-token\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.367419 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.367384 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-workload-socket\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.367454 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.367415 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-credential-socket\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.367454 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.367438 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-istio-data\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.367520 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.367454 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/086f87d9-dafc-4c6d-b660-d175a2723122-istio-podinfo\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.367555 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.367523 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-workload-certs\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.468433 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.468394 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/086f87d9-dafc-4c6d-b660-d175a2723122-istiod-ca-cert\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.468614 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.468442 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chd7r\" (UniqueName: \"kubernetes.io/projected/086f87d9-dafc-4c6d-b660-d175a2723122-kube-api-access-chd7r\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.468614 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.468488 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/086f87d9-dafc-4c6d-b660-d175a2723122-istio-token\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.468614 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.468507 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-workload-socket\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.468614 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.468535 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-credential-socket\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.468614 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.468569 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-istio-data\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.468887 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.468594 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/086f87d9-dafc-4c6d-b660-d175a2723122-istio-podinfo\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.468887 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.468708 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-workload-certs\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.468887 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.468738 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-istio-envoy\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.469207 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.469174 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-workload-certs\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.469303 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.469210 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-istio-data\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.469303 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.469258 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-credential-socket\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.469303 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.469280 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-workload-socket\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.469761 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.469737 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/086f87d9-dafc-4c6d-b660-d175a2723122-istiod-ca-cert\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.471062 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.471032 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-istio-envoy\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.471339 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.471316 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/086f87d9-dafc-4c6d-b660-d175a2723122-istio-podinfo\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.478027 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.477998 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/086f87d9-dafc-4c6d-b660-d175a2723122-istio-token\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.478160 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.478096 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chd7r\" (UniqueName: \"kubernetes.io/projected/086f87d9-dafc-4c6d-b660-d175a2723122-kube-api-access-chd7r\") pod \"e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.535660 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.535629 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:03:59.550311 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.550284 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:03:59.699285 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.699258 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd"] Jul 14 14:03:59.701412 ip-10-0-142-94 kubenswrapper[2583]: W0714 14:03:59.701371 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod086f87d9_dafc_4c6d_b660_d175a2723122.slice/crio-56df058bde097c6b1f2959c07623c6274073d08eb19f0f0f1fc3e6fd56689282 WatchSource:0}: Error finding container 56df058bde097c6b1f2959c07623c6274073d08eb19f0f0f1fc3e6fd56689282: Status 404 returned error can't find the container with id 56df058bde097c6b1f2959c07623c6274073d08eb19f0f0f1fc3e6fd56689282 Jul 14 14:03:59.707164 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.707117 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:03:59.707278 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.707197 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:03:59.707278 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:03:59.707228 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:04:00.159613 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:00.159575 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" event={"ID":"086f87d9-dafc-4c6d-b660-d175a2723122","Type":"ContainerStarted","Data":"867593a794270c78629616423aeb4c632d99ba826ca9ff29c817e70d314f3d34"} Jul 14 14:04:00.159814 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:00.159631 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" event={"ID":"086f87d9-dafc-4c6d-b660-d175a2723122","Type":"ContainerStarted","Data":"56df058bde097c6b1f2959c07623c6274073d08eb19f0f0f1fc3e6fd56689282"} Jul 14 14:04:00.184675 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:00.184626 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" podStartSLOduration=1.184609044 podStartE2EDuration="1.184609044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:03:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:04:00.182244753 +0000 UTC m=+2203.709395389" watchObservedRunningTime="2026-07-14 14:04:00.184609044 +0000 UTC m=+2203.711759739" Jul 14 14:04:00.551142 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:00.551076 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:04:00.556704 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:00.556676 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:04:01.165651 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:01.165613 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:04:01.166575 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:01.166553 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:04:02.193142 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:02.193103 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:02.275110 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:02.275053 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:02.342021 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:02.341980 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:02.533278 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:02.533247 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:02.850272 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:02.850190 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:04.449753 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:04.449717 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:06.622333 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:06.622298 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:06.941833 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:06.941787 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:07.240373 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.240289 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv"] Jul 14 14:04:07.244039 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.244015 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.247790 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.247766 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-f1dea9-openshift-default-dockercfg-bxkff\"" Jul 14 14:04:07.261527 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.261499 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv"] Jul 14 14:04:07.307024 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.306983 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:07.343548 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.343505 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-credential-socket\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.343743 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.343626 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-token\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.343743 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.343680 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpjm4\" (UniqueName: \"kubernetes.io/projected/1d557038-71f8-4ad4-b5f4-00bd512d035e-kube-api-access-fpjm4\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.343743 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.343729 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-workload-socket\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.343997 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.343781 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-podinfo\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.343997 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.343828 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-data\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.343997 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.343860 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d557038-71f8-4ad4-b5f4-00bd512d035e-istiod-ca-cert\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.343997 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.343935 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-workload-certs\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.343997 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.343994 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-envoy\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.444884 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.444840 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-workload-certs\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.444884 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.444890 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-envoy\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.445173 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.444913 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-credential-socket\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.445173 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.444940 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-token\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.445173 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.444962 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpjm4\" (UniqueName: \"kubernetes.io/projected/1d557038-71f8-4ad4-b5f4-00bd512d035e-kube-api-access-fpjm4\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.445173 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.445002 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-workload-socket\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.445173 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.445037 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-podinfo\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.445413 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.445386 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-workload-certs\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.445468 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.445410 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-credential-socket\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.445509 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.445473 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-workload-socket\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.445509 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.445493 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-data\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.445580 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.445535 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d557038-71f8-4ad4-b5f4-00bd512d035e-istiod-ca-cert\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.445740 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.445715 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-data\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.446185 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.446163 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d557038-71f8-4ad4-b5f4-00bd512d035e-istiod-ca-cert\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.447445 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.447421 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-envoy\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.447687 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.447668 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-podinfo\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.455430 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.455405 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpjm4\" (UniqueName: \"kubernetes.io/projected/1d557038-71f8-4ad4-b5f4-00bd512d035e-kube-api-access-fpjm4\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.455683 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.455657 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-token\") pod \"e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.474749 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.474723 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:07.555367 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.555329 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:07.698847 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.698818 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv"] Jul 14 14:04:07.701004 ip-10-0-142-94 kubenswrapper[2583]: W0714 14:04:07.700980 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d557038_71f8_4ad4_b5f4_00bd512d035e.slice/crio-6b5b5490a4726f4f42792b8c621b2edf4f9b8789160d77710e146ded0c0e1820 WatchSource:0}: Error finding container 6b5b5490a4726f4f42792b8c621b2edf4f9b8789160d77710e146ded0c0e1820: Status 404 returned error can't find the container with id 6b5b5490a4726f4f42792b8c621b2edf4f9b8789160d77710e146ded0c0e1820 Jul 14 14:04:07.703837 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.703800 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:04:07.703933 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.703880 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:04:07.703933 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:07.703925 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:04:08.201523 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:08.201488 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" event={"ID":"1d557038-71f8-4ad4-b5f4-00bd512d035e","Type":"ContainerStarted","Data":"5f191772fa74f74380fac324d447e1181083f06690f0c6aa577c3815bd832804"} Jul 14 14:04:08.201692 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:08.201538 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" event={"ID":"1d557038-71f8-4ad4-b5f4-00bd512d035e","Type":"ContainerStarted","Data":"6b5b5490a4726f4f42792b8c621b2edf4f9b8789160d77710e146ded0c0e1820"} Jul 14 14:04:08.225244 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:08.225193 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" podStartSLOduration=1.225176493 podStartE2EDuration="1.225176493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:04:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:04:08.223133389 +0000 UTC m=+2211.750284026" watchObservedRunningTime="2026-07-14 14:04:08.225176493 +0000 UTC m=+2211.752327129" Jul 14 14:04:08.555770 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:08.555727 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:08.560682 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:08.560654 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:09.206073 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:09.206041 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:09.207128 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:09.207106 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:10.156628 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:10.156592 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:10.261699 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:10.261656 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:10.558456 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:10.558416 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:10.660508 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:10.660473 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:10.762221 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:10.762179 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:10.924855 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:10.924699 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:14.655620 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:14.655589 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:15.340368 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:15.340329 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:17.207316 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:17.207259 2583 patch_prober.go:28] interesting pod/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 14 14:04:17.207807 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:17.207341 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" podUID="086f87d9-dafc-4c6d-b660-d175a2723122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 14 14:04:19.267558 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:19.267524 2583 generic.go:356] "Generic (PLEG): container finished" podID="086f87d9-dafc-4c6d-b660-d175a2723122" containerID="867593a794270c78629616423aeb4c632d99ba826ca9ff29c817e70d314f3d34" exitCode=0 Jul 14 14:04:19.268105 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:19.267606 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" event={"ID":"086f87d9-dafc-4c6d-b660-d175a2723122","Type":"ContainerDied","Data":"867593a794270c78629616423aeb4c632d99ba826ca9ff29c817e70d314f3d34"} Jul 14 14:04:19.268178 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:19.268163 2583 scope.go:117] "RemoveContainer" containerID="867593a794270c78629616423aeb4c632d99ba826ca9ff29c817e70d314f3d34" Jul 14 14:04:19.269198 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:19.269168 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:04:19.269281 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:19.269235 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:04:19.269281 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:19.269266 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:04:20.274615 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:20.274579 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" event={"ID":"086f87d9-dafc-4c6d-b660-d175a2723122","Type":"ContainerStarted","Data":"9929ec75a7042ae432443ed2bc908ef94f2e173b37ac5784e0cd78c9eafd13de"} Jul 14 14:04:20.551325 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:20.551218 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:04:20.552029 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:20.551635 2583 patch_prober.go:28] interesting pod/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 14 14:04:20.552029 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:20.551709 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" podUID="086f87d9-dafc-4c6d-b660-d175a2723122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 14 14:04:21.551631 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:21.551590 2583 patch_prober.go:28] interesting pod/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 14 14:04:21.552268 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:21.552227 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" podUID="086f87d9-dafc-4c6d-b660-d175a2723122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 14 14:04:22.551506 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:22.551466 2583 patch_prober.go:28] interesting pod/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 14 14:04:22.551835 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:22.551803 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" podUID="086f87d9-dafc-4c6d-b660-d175a2723122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 14 14:04:23.551279 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:23.551229 2583 patch_prober.go:28] interesting pod/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 14 14:04:23.551707 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:23.551649 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" podUID="086f87d9-dafc-4c6d-b660-d175a2723122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 14 14:04:23.884416 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:23.884339 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:24.555335 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:24.555302 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:04:24.555587 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:24.555565 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:04:24.556234 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:24.556215 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:04:25.786810 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:25.786779 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:27.312638 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:27.312599 2583 generic.go:356] "Generic (PLEG): container finished" podID="1d557038-71f8-4ad4-b5f4-00bd512d035e" containerID="5f191772fa74f74380fac324d447e1181083f06690f0c6aa577c3815bd832804" exitCode=0 Jul 14 14:04:27.313143 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:27.312669 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" event={"ID":"1d557038-71f8-4ad4-b5f4-00bd512d035e","Type":"ContainerDied","Data":"5f191772fa74f74380fac324d447e1181083f06690f0c6aa577c3815bd832804"} Jul 14 14:04:27.313143 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:27.313071 2583 scope.go:117] "RemoveContainer" containerID="5f191772fa74f74380fac324d447e1181083f06690f0c6aa577c3815bd832804" Jul 14 14:04:27.314127 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:27.314097 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:04:27.314239 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:27.314167 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:04:27.314239 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:27.314198 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:04:28.319997 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:28.319963 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" event={"ID":"1d557038-71f8-4ad4-b5f4-00bd512d035e","Type":"ContainerStarted","Data":"22aa256afe4ae5d645ba3377fd6b7e7b02a785020ded9d6db4cbeb95157816ce"} Jul 14 14:04:28.556300 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:28.556266 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:28.556566 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:28.556541 2583 patch_prober.go:28] interesting pod/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 14 14:04:28.556641 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:28.556595 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" podUID="1d557038-71f8-4ad4-b5f4-00bd512d035e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 14 14:04:29.556539 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:29.556503 2583 patch_prober.go:28] interesting pod/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 14 14:04:29.556900 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:29.556563 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" podUID="1d557038-71f8-4ad4-b5f4-00bd512d035e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 14 14:04:30.555862 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:30.555824 2583 patch_prober.go:28] interesting pod/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 14 14:04:30.556042 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:30.555898 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" podUID="1d557038-71f8-4ad4-b5f4-00bd512d035e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 14 14:04:31.555864 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:31.555824 2583 patch_prober.go:28] interesting pod/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 14 14:04:31.556368 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:31.555893 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" podUID="1d557038-71f8-4ad4-b5f4-00bd512d035e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 14 14:04:32.476637 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:32.476595 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:32.559791 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:32.559758 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:32.560205 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:32.560183 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:32.560855 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:32.560837 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:04:32.572460 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:32.572434 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:04:37.509766 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:04:37.509734 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:05:29.518703 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:05:29.518653 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:05:42.706396 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:05:42.706359 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:05:42.808278 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:05:42.808241 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:05.242619 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:05.242581 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:06.237057 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:06.237027 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:06.859842 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:06.859804 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:06.960951 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:06.960901 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:16.899705 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:16.899615 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:21.464427 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:21.464393 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:27.762611 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:27.762569 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:27.829461 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:27.829415 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:32.131761 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:32.131731 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd"] Jul 14 14:06:32.132181 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:32.131961 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" podUID="086f87d9-dafc-4c6d-b660-d175a2723122" containerName="istio-proxy" containerID="cri-o://9929ec75a7042ae432443ed2bc908ef94f2e173b37ac5784e0cd78c9eafd13de" gracePeriod=30 Jul 14 14:06:32.155767 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:32.155739 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:32.405393 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:32.405294 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:33.027717 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:33.027680 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:33.127319 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:33.127276 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:37.395150 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.395128 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:06:37.491891 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.491811 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/086f87d9-dafc-4c6d-b660-d175a2723122-istio-token\") pod \"086f87d9-dafc-4c6d-b660-d175a2723122\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " Jul 14 14:06:37.491891 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.491864 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-workload-certs\") pod \"086f87d9-dafc-4c6d-b660-d175a2723122\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " Jul 14 14:06:37.492136 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.491899 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-istio-data\") pod \"086f87d9-dafc-4c6d-b660-d175a2723122\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " Jul 14 14:06:37.492136 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.492011 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-workload-socket\") pod \"086f87d9-dafc-4c6d-b660-d175a2723122\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " Jul 14 14:06:37.492260 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.492164 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/086f87d9-dafc-4c6d-b660-d175a2723122-istiod-ca-cert\") pod \"086f87d9-dafc-4c6d-b660-d175a2723122\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " Jul 14 14:06:37.492260 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.492204 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-istio-envoy\") pod \"086f87d9-dafc-4c6d-b660-d175a2723122\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " Jul 14 14:06:37.492260 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.492237 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-credential-socket\") pod \"086f87d9-dafc-4c6d-b660-d175a2723122\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " Jul 14 14:06:37.492260 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.492233 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "086f87d9-dafc-4c6d-b660-d175a2723122" (UID: "086f87d9-dafc-4c6d-b660-d175a2723122"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:06:37.492454 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.492273 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "086f87d9-dafc-4c6d-b660-d175a2723122" (UID: "086f87d9-dafc-4c6d-b660-d175a2723122"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:06:37.492454 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.492289 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chd7r\" (UniqueName: \"kubernetes.io/projected/086f87d9-dafc-4c6d-b660-d175a2723122-kube-api-access-chd7r\") pod \"086f87d9-dafc-4c6d-b660-d175a2723122\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " Jul 14 14:06:37.492454 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.492316 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/086f87d9-dafc-4c6d-b660-d175a2723122-istio-podinfo\") pod \"086f87d9-dafc-4c6d-b660-d175a2723122\" (UID: \"086f87d9-dafc-4c6d-b660-d175a2723122\") " Jul 14 14:06:37.492454 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.492439 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "086f87d9-dafc-4c6d-b660-d175a2723122" (UID: "086f87d9-dafc-4c6d-b660-d175a2723122"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:06:37.492664 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.492543 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/086f87d9-dafc-4c6d-b660-d175a2723122-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "086f87d9-dafc-4c6d-b660-d175a2723122" (UID: "086f87d9-dafc-4c6d-b660-d175a2723122"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:06:37.492724 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.492677 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-workload-certs\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:06:37.492724 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.492698 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-workload-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:06:37.492724 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.492713 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/086f87d9-dafc-4c6d-b660-d175a2723122-istiod-ca-cert\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:06:37.492822 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.492727 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-credential-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:06:37.492905 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.492882 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-istio-data" (OuterVolumeSpecName: "istio-data") pod "086f87d9-dafc-4c6d-b660-d175a2723122" (UID: "086f87d9-dafc-4c6d-b660-d175a2723122"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:06:37.494304 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.494284 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "086f87d9-dafc-4c6d-b660-d175a2723122" (UID: "086f87d9-dafc-4c6d-b660-d175a2723122"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:06:37.494435 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.494419 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/086f87d9-dafc-4c6d-b660-d175a2723122-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "086f87d9-dafc-4c6d-b660-d175a2723122" (UID: "086f87d9-dafc-4c6d-b660-d175a2723122"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 14:06:37.494539 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.494519 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/086f87d9-dafc-4c6d-b660-d175a2723122-istio-token" (OuterVolumeSpecName: "istio-token") pod "086f87d9-dafc-4c6d-b660-d175a2723122" (UID: "086f87d9-dafc-4c6d-b660-d175a2723122"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:06:37.494621 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.494599 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/086f87d9-dafc-4c6d-b660-d175a2723122-kube-api-access-chd7r" (OuterVolumeSpecName: "kube-api-access-chd7r") pod "086f87d9-dafc-4c6d-b660-d175a2723122" (UID: "086f87d9-dafc-4c6d-b660-d175a2723122"). InnerVolumeSpecName "kube-api-access-chd7r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:06:37.594188 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.594151 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-istio-envoy\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:06:37.594188 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.594184 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chd7r\" (UniqueName: \"kubernetes.io/projected/086f87d9-dafc-4c6d-b660-d175a2723122-kube-api-access-chd7r\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:06:37.594387 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.594198 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/086f87d9-dafc-4c6d-b660-d175a2723122-istio-podinfo\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:06:37.594387 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.594210 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/086f87d9-dafc-4c6d-b660-d175a2723122-istio-token\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:06:37.594387 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.594221 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/086f87d9-dafc-4c6d-b660-d175a2723122-istio-data\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:06:37.976790 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.976758 2583 generic.go:356] "Generic (PLEG): container finished" podID="086f87d9-dafc-4c6d-b660-d175a2723122" containerID="9929ec75a7042ae432443ed2bc908ef94f2e173b37ac5784e0cd78c9eafd13de" exitCode=0 Jul 14 14:06:37.976971 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.976816 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" Jul 14 14:06:37.976971 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.976831 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" event={"ID":"086f87d9-dafc-4c6d-b660-d175a2723122","Type":"ContainerDied","Data":"9929ec75a7042ae432443ed2bc908ef94f2e173b37ac5784e0cd78c9eafd13de"} Jul 14 14:06:37.976971 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.976875 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd" event={"ID":"086f87d9-dafc-4c6d-b660-d175a2723122","Type":"ContainerDied","Data":"56df058bde097c6b1f2959c07623c6274073d08eb19f0f0f1fc3e6fd56689282"} Jul 14 14:06:37.976971 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.976897 2583 scope.go:117] "RemoveContainer" containerID="9929ec75a7042ae432443ed2bc908ef94f2e173b37ac5784e0cd78c9eafd13de" Jul 14 14:06:37.986215 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.986199 2583 scope.go:117] "RemoveContainer" containerID="867593a794270c78629616423aeb4c632d99ba826ca9ff29c817e70d314f3d34" Jul 14 14:06:37.994390 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.994372 2583 scope.go:117] "RemoveContainer" containerID="9929ec75a7042ae432443ed2bc908ef94f2e173b37ac5784e0cd78c9eafd13de" Jul 14 14:06:37.994646 ip-10-0-142-94 kubenswrapper[2583]: E0714 14:06:37.994626 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9929ec75a7042ae432443ed2bc908ef94f2e173b37ac5784e0cd78c9eafd13de\": container with ID starting with 9929ec75a7042ae432443ed2bc908ef94f2e173b37ac5784e0cd78c9eafd13de not found: ID does not exist" containerID="9929ec75a7042ae432443ed2bc908ef94f2e173b37ac5784e0cd78c9eafd13de" Jul 14 14:06:37.994694 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.994655 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9929ec75a7042ae432443ed2bc908ef94f2e173b37ac5784e0cd78c9eafd13de"} err="failed to get container status \"9929ec75a7042ae432443ed2bc908ef94f2e173b37ac5784e0cd78c9eafd13de\": rpc error: code = NotFound desc = could not find container \"9929ec75a7042ae432443ed2bc908ef94f2e173b37ac5784e0cd78c9eafd13de\": container with ID starting with 9929ec75a7042ae432443ed2bc908ef94f2e173b37ac5784e0cd78c9eafd13de not found: ID does not exist" Jul 14 14:06:37.994694 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.994673 2583 scope.go:117] "RemoveContainer" containerID="867593a794270c78629616423aeb4c632d99ba826ca9ff29c817e70d314f3d34" Jul 14 14:06:37.994882 ip-10-0-142-94 kubenswrapper[2583]: E0714 14:06:37.994866 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"867593a794270c78629616423aeb4c632d99ba826ca9ff29c817e70d314f3d34\": container with ID starting with 867593a794270c78629616423aeb4c632d99ba826ca9ff29c817e70d314f3d34 not found: ID does not exist" containerID="867593a794270c78629616423aeb4c632d99ba826ca9ff29c817e70d314f3d34" Jul 14 14:06:37.994951 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:37.994885 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"867593a794270c78629616423aeb4c632d99ba826ca9ff29c817e70d314f3d34"} err="failed to get container status \"867593a794270c78629616423aeb4c632d99ba826ca9ff29c817e70d314f3d34\": rpc error: code = NotFound desc = could not find container \"867593a794270c78629616423aeb4c632d99ba826ca9ff29c817e70d314f3d34\": container with ID starting with 867593a794270c78629616423aeb4c632d99ba826ca9ff29c817e70d314f3d34 not found: ID does not exist" Jul 14 14:06:38.000809 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:38.000785 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd"] Jul 14 14:06:38.005115 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:38.005093 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-370ee8-openshift-default-67c98d8c9c-2zppd"] Jul 14 14:06:38.494205 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:38.494166 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="086f87d9-dafc-4c6d-b660-d175a2723122" path="/var/lib/kubelet/pods/086f87d9-dafc-4c6d-b660-d175a2723122/volumes" Jul 14 14:06:42.995620 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:42.995581 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:47.919215 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:47.919179 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:53.745514 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:53.745469 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:53.846493 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:53.846440 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:58.262298 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:58.262263 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv"] Jul 14 14:06:58.262811 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:58.262547 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" podUID="1d557038-71f8-4ad4-b5f4-00bd512d035e" containerName="istio-proxy" containerID="cri-o://22aa256afe4ae5d645ba3377fd6b7e7b02a785020ded9d6db4cbeb95157816ce" gracePeriod=30 Jul 14 14:06:58.280811 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:58.280776 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:58.502653 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:58.502613 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:58.869134 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:58.869062 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:59.129911 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.129829 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd"] Jul 14 14:06:59.130631 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.130607 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="086f87d9-dafc-4c6d-b660-d175a2723122" containerName="istio-proxy" Jul 14 14:06:59.130738 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.130635 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="086f87d9-dafc-4c6d-b660-d175a2723122" containerName="istio-proxy" Jul 14 14:06:59.130738 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.130659 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="086f87d9-dafc-4c6d-b660-d175a2723122" containerName="istio-proxy" Jul 14 14:06:59.130738 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.130668 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="086f87d9-dafc-4c6d-b660-d175a2723122" containerName="istio-proxy" Jul 14 14:06:59.130933 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.130916 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="086f87d9-dafc-4c6d-b660-d175a2723122" containerName="istio-proxy" Jul 14 14:06:59.130982 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.130971 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="086f87d9-dafc-4c6d-b660-d175a2723122" containerName="istio-proxy" Jul 14 14:06:59.133880 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.133861 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.136916 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.136886 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-f2a083-openshift-default-dockercfg-r2xxd\"" Jul 14 14:06:59.145842 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.145819 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd"] Jul 14 14:06:59.154915 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.154885 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:59.283472 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.283438 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:06:59.305320 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.305273 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istiod-ca-cert\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.305469 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.305341 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-data\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.305469 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.305380 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-podinfo\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.305469 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.305418 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-envoy\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.305469 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.305437 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-workload-certs\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.305469 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.305460 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzvnt\" (UniqueName: \"kubernetes.io/projected/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-kube-api-access-fzvnt\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.305635 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.305513 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-token\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.305635 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.305563 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-workload-socket\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.305635 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.305584 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-credential-socket\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.406537 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.406440 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istiod-ca-cert\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.406537 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.406495 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-data\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.406727 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.406606 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-podinfo\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.406727 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.406667 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-envoy\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.406727 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.406697 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-workload-certs\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.406865 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.406729 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzvnt\" (UniqueName: \"kubernetes.io/projected/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-kube-api-access-fzvnt\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.406865 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.406763 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-token\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.406865 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.406799 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-workload-socket\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.406865 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.406824 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-credential-socket\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.407059 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.406862 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-data\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.407268 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.407245 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-workload-socket\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.407448 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.407283 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-credential-socket\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.407448 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.407292 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-workload-certs\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.407564 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.407315 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istiod-ca-cert\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.408982 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.408956 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-envoy\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.409240 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.409219 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-podinfo\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.419577 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.419550 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-token\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.419768 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.419743 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzvnt\" (UniqueName: \"kubernetes.io/projected/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-kube-api-access-fzvnt\") pod \"e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.445735 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.445705 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:06:59.591994 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.591957 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd"] Jul 14 14:06:59.595232 ip-10-0-142-94 kubenswrapper[2583]: W0714 14:06:59.595200 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fb7b26b_17fd_4c7a_a0db_b004ae1d0cfb.slice/crio-136f2d33fd1e9167ea7a3393be15ef844d5ea62dc1705b1877444a89c8c873b0 WatchSource:0}: Error finding container 136f2d33fd1e9167ea7a3393be15ef844d5ea62dc1705b1877444a89c8c873b0: Status 404 returned error can't find the container with id 136f2d33fd1e9167ea7a3393be15ef844d5ea62dc1705b1877444a89c8c873b0 Jul 14 14:06:59.597329 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.597297 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:06:59.597409 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.597364 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:06:59.597409 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:06:59.597391 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:07:00.089313 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:00.089267 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" event={"ID":"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb","Type":"ContainerStarted","Data":"2dc141ccbd3a8c444db463093fb47d7888b5b467558676cfbae59eaf4daba246"} Jul 14 14:07:00.089519 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:00.089332 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" event={"ID":"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb","Type":"ContainerStarted","Data":"136f2d33fd1e9167ea7a3393be15ef844d5ea62dc1705b1877444a89c8c873b0"} Jul 14 14:07:00.114298 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:00.114245 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" podStartSLOduration=1.114230627 podStartE2EDuration="1.114230627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:06:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:07:00.112364925 +0000 UTC m=+2383.639515562" watchObservedRunningTime="2026-07-14 14:07:00.114230627 +0000 UTC m=+2383.641381263" Jul 14 14:07:00.446314 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:00.446219 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:07:00.451594 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:00.451564 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:07:01.094241 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:01.094204 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:07:01.095353 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:01.095325 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:07:02.022263 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:02.022230 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:02.149098 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:02.149052 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:02.257887 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:02.257848 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:02.500692 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:02.500664 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:02.562883 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:02.562845 2583 patch_prober.go:28] interesting pod/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 14:07:02.563071 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:02.562919 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" podUID="1d557038-71f8-4ad4-b5f4-00bd512d035e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 14:07:02.607004 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:02.606967 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:02.749873 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:02.749840 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:03.535559 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.535483 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:07:03.653497 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.653464 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-token\") pod \"1d557038-71f8-4ad4-b5f4-00bd512d035e\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " Jul 14 14:07:03.653497 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.653502 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-data\") pod \"1d557038-71f8-4ad4-b5f4-00bd512d035e\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " Jul 14 14:07:03.653753 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.653538 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-workload-certs\") pod \"1d557038-71f8-4ad4-b5f4-00bd512d035e\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " Jul 14 14:07:03.653753 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.653574 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-podinfo\") pod \"1d557038-71f8-4ad4-b5f4-00bd512d035e\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " Jul 14 14:07:03.653753 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.653602 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-envoy\") pod \"1d557038-71f8-4ad4-b5f4-00bd512d035e\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " Jul 14 14:07:03.653753 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.653659 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-workload-socket\") pod \"1d557038-71f8-4ad4-b5f4-00bd512d035e\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " Jul 14 14:07:03.653753 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.653711 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d557038-71f8-4ad4-b5f4-00bd512d035e-istiod-ca-cert\") pod \"1d557038-71f8-4ad4-b5f4-00bd512d035e\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " Jul 14 14:07:03.653753 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.653736 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-credential-socket\") pod \"1d557038-71f8-4ad4-b5f4-00bd512d035e\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " Jul 14 14:07:03.654159 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.653773 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpjm4\" (UniqueName: \"kubernetes.io/projected/1d557038-71f8-4ad4-b5f4-00bd512d035e-kube-api-access-fpjm4\") pod \"1d557038-71f8-4ad4-b5f4-00bd512d035e\" (UID: \"1d557038-71f8-4ad4-b5f4-00bd512d035e\") " Jul 14 14:07:03.654159 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.653880 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1d557038-71f8-4ad4-b5f4-00bd512d035e" (UID: "1d557038-71f8-4ad4-b5f4-00bd512d035e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:07:03.654159 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.654048 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1d557038-71f8-4ad4-b5f4-00bd512d035e" (UID: "1d557038-71f8-4ad4-b5f4-00bd512d035e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:07:03.654319 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.654176 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-credential-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:07:03.654319 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.654178 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d557038-71f8-4ad4-b5f4-00bd512d035e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1d557038-71f8-4ad4-b5f4-00bd512d035e" (UID: "1d557038-71f8-4ad4-b5f4-00bd512d035e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:07:03.654319 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.654189 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1d557038-71f8-4ad4-b5f4-00bd512d035e" (UID: "1d557038-71f8-4ad4-b5f4-00bd512d035e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:07:03.654319 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.654195 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-workload-certs\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:07:03.654504 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.654481 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-data" (OuterVolumeSpecName: "istio-data") pod "1d557038-71f8-4ad4-b5f4-00bd512d035e" (UID: "1d557038-71f8-4ad4-b5f4-00bd512d035e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:07:03.655924 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.655893 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1d557038-71f8-4ad4-b5f4-00bd512d035e" (UID: "1d557038-71f8-4ad4-b5f4-00bd512d035e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 14:07:03.656034 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.655939 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-token" (OuterVolumeSpecName: "istio-token") pod "1d557038-71f8-4ad4-b5f4-00bd512d035e" (UID: "1d557038-71f8-4ad4-b5f4-00bd512d035e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:07:03.656034 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.656002 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d557038-71f8-4ad4-b5f4-00bd512d035e-kube-api-access-fpjm4" (OuterVolumeSpecName: "kube-api-access-fpjm4") pod "1d557038-71f8-4ad4-b5f4-00bd512d035e" (UID: "1d557038-71f8-4ad4-b5f4-00bd512d035e"). InnerVolumeSpecName "kube-api-access-fpjm4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:07:03.656272 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.656250 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1d557038-71f8-4ad4-b5f4-00bd512d035e" (UID: "1d557038-71f8-4ad4-b5f4-00bd512d035e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:07:03.755528 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.755486 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-token\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:07:03.755528 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.755521 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-data\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:07:03.755528 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.755532 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-podinfo\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:07:03.755780 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.755544 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-istio-envoy\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:07:03.755780 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.755555 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d557038-71f8-4ad4-b5f4-00bd512d035e-workload-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:07:03.755780 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.755566 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d557038-71f8-4ad4-b5f4-00bd512d035e-istiod-ca-cert\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:07:03.755780 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:03.755578 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpjm4\" (UniqueName: \"kubernetes.io/projected/1d557038-71f8-4ad4-b5f4-00bd512d035e-kube-api-access-fpjm4\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:07:04.117754 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:04.117576 2583 generic.go:356] "Generic (PLEG): container finished" podID="1d557038-71f8-4ad4-b5f4-00bd512d035e" containerID="22aa256afe4ae5d645ba3377fd6b7e7b02a785020ded9d6db4cbeb95157816ce" exitCode=0 Jul 14 14:07:04.117969 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:04.117766 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" event={"ID":"1d557038-71f8-4ad4-b5f4-00bd512d035e","Type":"ContainerDied","Data":"22aa256afe4ae5d645ba3377fd6b7e7b02a785020ded9d6db4cbeb95157816ce"} Jul 14 14:07:04.117969 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:04.117851 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" event={"ID":"1d557038-71f8-4ad4-b5f4-00bd512d035e","Type":"ContainerDied","Data":"6b5b5490a4726f4f42792b8c621b2edf4f9b8789160d77710e146ded0c0e1820"} Jul 14 14:07:04.117969 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:04.117891 2583 scope.go:117] "RemoveContainer" containerID="22aa256afe4ae5d645ba3377fd6b7e7b02a785020ded9d6db4cbeb95157816ce" Jul 14 14:07:04.118166 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:04.118127 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv" Jul 14 14:07:04.129115 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:04.128919 2583 scope.go:117] "RemoveContainer" containerID="5f191772fa74f74380fac324d447e1181083f06690f0c6aa577c3815bd832804" Jul 14 14:07:04.139474 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:04.139443 2583 scope.go:117] "RemoveContainer" containerID="22aa256afe4ae5d645ba3377fd6b7e7b02a785020ded9d6db4cbeb95157816ce" Jul 14 14:07:04.139817 ip-10-0-142-94 kubenswrapper[2583]: E0714 14:07:04.139793 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22aa256afe4ae5d645ba3377fd6b7e7b02a785020ded9d6db4cbeb95157816ce\": container with ID starting with 22aa256afe4ae5d645ba3377fd6b7e7b02a785020ded9d6db4cbeb95157816ce not found: ID does not exist" containerID="22aa256afe4ae5d645ba3377fd6b7e7b02a785020ded9d6db4cbeb95157816ce" Jul 14 14:07:04.139919 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:04.139825 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22aa256afe4ae5d645ba3377fd6b7e7b02a785020ded9d6db4cbeb95157816ce"} err="failed to get container status \"22aa256afe4ae5d645ba3377fd6b7e7b02a785020ded9d6db4cbeb95157816ce\": rpc error: code = NotFound desc = could not find container \"22aa256afe4ae5d645ba3377fd6b7e7b02a785020ded9d6db4cbeb95157816ce\": container with ID starting with 22aa256afe4ae5d645ba3377fd6b7e7b02a785020ded9d6db4cbeb95157816ce not found: ID does not exist" Jul 14 14:07:04.139919 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:04.139853 2583 scope.go:117] "RemoveContainer" containerID="5f191772fa74f74380fac324d447e1181083f06690f0c6aa577c3815bd832804" Jul 14 14:07:04.140190 ip-10-0-142-94 kubenswrapper[2583]: E0714 14:07:04.140167 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f191772fa74f74380fac324d447e1181083f06690f0c6aa577c3815bd832804\": container with ID starting with 5f191772fa74f74380fac324d447e1181083f06690f0c6aa577c3815bd832804 not found: ID does not exist" containerID="5f191772fa74f74380fac324d447e1181083f06690f0c6aa577c3815bd832804" Jul 14 14:07:04.140266 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:04.140200 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f191772fa74f74380fac324d447e1181083f06690f0c6aa577c3815bd832804"} err="failed to get container status \"5f191772fa74f74380fac324d447e1181083f06690f0c6aa577c3815bd832804\": rpc error: code = NotFound desc = could not find container \"5f191772fa74f74380fac324d447e1181083f06690f0c6aa577c3815bd832804\": container with ID starting with 5f191772fa74f74380fac324d447e1181083f06690f0c6aa577c3815bd832804 not found: ID does not exist" Jul 14 14:07:04.147762 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:04.147731 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv"] Jul 14 14:07:04.153695 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:04.153669 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-f1dea9-openshift-default-5d878f78bc-xg6mv"] Jul 14 14:07:04.493342 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:04.493250 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d557038-71f8-4ad4-b5f4-00bd512d035e" path="/var/lib/kubelet/pods/1d557038-71f8-4ad4-b5f4-00bd512d035e/volumes" Jul 14 14:07:06.524772 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:06.524721 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:06.999487 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:06.999454 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:07.300263 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.300230 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt"] Jul 14 14:07:07.300704 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.300689 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d557038-71f8-4ad4-b5f4-00bd512d035e" containerName="istio-proxy" Jul 14 14:07:07.300704 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.300706 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d557038-71f8-4ad4-b5f4-00bd512d035e" containerName="istio-proxy" Jul 14 14:07:07.300792 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.300718 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d557038-71f8-4ad4-b5f4-00bd512d035e" containerName="istio-proxy" Jul 14 14:07:07.300792 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.300724 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d557038-71f8-4ad4-b5f4-00bd512d035e" containerName="istio-proxy" Jul 14 14:07:07.300862 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.300795 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d557038-71f8-4ad4-b5f4-00bd512d035e" containerName="istio-proxy" Jul 14 14:07:07.300862 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.300807 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d557038-71f8-4ad4-b5f4-00bd512d035e" containerName="istio-proxy" Jul 14 14:07:07.305416 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.305392 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.308282 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.308258 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-1816dc-openshift-default-dockercfg-ctz6d\"" Jul 14 14:07:07.316668 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.316641 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt"] Jul 14 14:07:07.332542 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.332510 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:07.431895 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.431864 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:07.491021 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.490989 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-data\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.491223 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.491032 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-workload-certs\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.491223 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.491057 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-workload-socket\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.491223 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.491169 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-token\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.491223 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.491213 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-podinfo\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.491365 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.491260 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px2rd\" (UniqueName: \"kubernetes.io/projected/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-kube-api-access-px2rd\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.491365 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.491328 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-envoy\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.491365 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.491350 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-credential-socket\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.491459 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.491375 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istiod-ca-cert\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.593029 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.592939 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px2rd\" (UniqueName: \"kubernetes.io/projected/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-kube-api-access-px2rd\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.593029 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.592985 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-envoy\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.593029 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.593006 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-credential-socket\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.593582 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.593133 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istiod-ca-cert\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.593582 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.593242 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-data\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.593582 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.593294 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-workload-certs\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.593582 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.593341 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-workload-socket\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.593582 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.593387 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-token\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.593582 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.593432 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-podinfo\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.593582 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.593439 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-credential-socket\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.593918 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.593621 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-data\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.593918 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.593653 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-workload-certs\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.593918 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.593741 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-workload-socket\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.593918 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.593867 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istiod-ca-cert\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.595451 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.595421 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-envoy\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.595751 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.595734 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-podinfo\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.601896 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.601868 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-token\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.602020 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.601993 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px2rd\" (UniqueName: \"kubernetes.io/projected/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-kube-api-access-px2rd\") pod \"e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.617802 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.617779 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:07.759867 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.759843 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt"] Jul 14 14:07:07.764799 ip-10-0-142-94 kubenswrapper[2583]: W0714 14:07:07.764750 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ef38dc3_c7bb_4d5c_b883_6d1aa227c926.slice/crio-a253adcad38c757d34b1cd8fa24c87204108579222b4cd4738d7eaa52ad96686 WatchSource:0}: Error finding container a253adcad38c757d34b1cd8fa24c87204108579222b4cd4738d7eaa52ad96686: Status 404 returned error can't find the container with id a253adcad38c757d34b1cd8fa24c87204108579222b4cd4738d7eaa52ad96686 Jul 14 14:07:07.766900 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.766860 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:07:07.766998 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.766943 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:07:07.766998 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:07.766985 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:07:08.146228 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:08.146189 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" event={"ID":"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926","Type":"ContainerStarted","Data":"11e02b2de2faed002bd86b261089b36d9a1b7f7de2d5bf423ec398accad67184"} Jul 14 14:07:08.146422 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:08.146260 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" event={"ID":"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926","Type":"ContainerStarted","Data":"a253adcad38c757d34b1cd8fa24c87204108579222b4cd4738d7eaa52ad96686"} Jul 14 14:07:08.168484 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:08.168435 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" podStartSLOduration=1.168418372 podStartE2EDuration="1.168418372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:07:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:07:08.167394724 +0000 UTC m=+2391.694545361" watchObservedRunningTime="2026-07-14 14:07:08.168418372 +0000 UTC m=+2391.695569008" Jul 14 14:07:08.618054 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:08.618024 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:08.622667 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:08.622644 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:09.151425 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:09.151397 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:09.152284 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:09.152265 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:10.108638 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:10.108609 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:10.228357 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:10.228322 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:10.360635 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:10.360544 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:10.553609 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:10.553570 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:11.094103 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:11.094039 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:11.190947 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:11.190908 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:14.613714 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:14.613680 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:17.136263 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:17.136221 2583 patch_prober.go:28] interesting pod/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 14 14:07:17.136662 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:17.136301 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" podUID="8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 14 14:07:18.197434 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:18.197398 2583 generic.go:356] "Generic (PLEG): container finished" podID="8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" containerID="2dc141ccbd3a8c444db463093fb47d7888b5b467558676cfbae59eaf4daba246" exitCode=0 Jul 14 14:07:18.198015 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:18.197479 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" event={"ID":"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb","Type":"ContainerDied","Data":"2dc141ccbd3a8c444db463093fb47d7888b5b467558676cfbae59eaf4daba246"} Jul 14 14:07:18.198111 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:18.198042 2583 scope.go:117] "RemoveContainer" containerID="2dc141ccbd3a8c444db463093fb47d7888b5b467558676cfbae59eaf4daba246" Jul 14 14:07:18.199188 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:18.199156 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:07:18.199304 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:18.199227 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:07:18.199304 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:18.199268 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:07:18.556486 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:18.556455 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 14:07:18.559492 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:18.559471 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 14:07:19.208103 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:19.208028 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" event={"ID":"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb","Type":"ContainerStarted","Data":"5bb78244f55bc98401d60a5a43ebceddad8661ce7501ef1d638d7bf087148b4e"} Jul 14 14:07:19.446721 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:19.446686 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:07:19.447112 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:19.447004 2583 patch_prober.go:28] interesting pod/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 14 14:07:19.447284 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:19.447256 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" podUID="8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 14 14:07:20.446804 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:20.446758 2583 patch_prober.go:28] interesting pod/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 14 14:07:20.447338 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:20.446832 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" podUID="8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 14 14:07:21.446502 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:21.446451 2583 patch_prober.go:28] interesting pod/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 14 14:07:21.446722 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:21.446524 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" podUID="8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 14 14:07:22.446923 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:22.446884 2583 patch_prober.go:28] interesting pod/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 14 14:07:22.447569 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:22.447522 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" podUID="8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 14 14:07:23.368282 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:23.368213 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:23.465025 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:23.464989 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:07:23.465528 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:23.465405 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:07:23.466592 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:23.466570 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:07:23.466711 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:23.466567 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:25.152047 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:25.152010 2583 patch_prober.go:28] interesting pod/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 14 14:07:25.152458 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:25.152064 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" podUID="1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 14 14:07:26.246822 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:26.246799 2583 generic.go:356] "Generic (PLEG): container finished" podID="1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" containerID="11e02b2de2faed002bd86b261089b36d9a1b7f7de2d5bf423ec398accad67184" exitCode=0 Jul 14 14:07:26.247173 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:26.246869 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" event={"ID":"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926","Type":"ContainerDied","Data":"11e02b2de2faed002bd86b261089b36d9a1b7f7de2d5bf423ec398accad67184"} Jul 14 14:07:26.247361 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:26.247346 2583 scope.go:117] "RemoveContainer" containerID="11e02b2de2faed002bd86b261089b36d9a1b7f7de2d5bf423ec398accad67184" Jul 14 14:07:26.248348 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:26.248324 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:07:26.248412 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:26.248390 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:07:26.248449 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:26.248420 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:07:27.255478 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:27.255438 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" event={"ID":"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926","Type":"ContainerStarted","Data":"985a4740b84041ed8dca211adf3745d3a9f5d35e5c7026e0664596ad1d34ec11"} Jul 14 14:07:27.618798 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:27.618699 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:27.618945 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:27.618915 2583 patch_prober.go:28] interesting pod/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 14 14:07:27.619023 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:27.619002 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" podUID="1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 14 14:07:28.618822 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:28.618790 2583 patch_prober.go:28] interesting pod/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 14 14:07:28.619232 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:28.618843 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" podUID="1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 14 14:07:29.619106 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:29.619036 2583 patch_prober.go:28] interesting pod/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 14 14:07:29.619590 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:29.619173 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" podUID="1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 14 14:07:30.618889 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:30.618854 2583 patch_prober.go:28] interesting pod/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 14 14:07:30.619069 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:30.618908 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" podUID="1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 14 14:07:31.622315 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:31.622280 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:31.622684 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:31.622616 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:31.623337 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:31.623320 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:07:32.634815 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:32.634772 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:32.729562 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:32.729521 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:32.885056 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:32.884932 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:37.810554 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:37.810517 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:43.980423 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:43.980322 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:44.051431 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:44.051385 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:44.972846 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:44.972801 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:48.462150 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:48.462115 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd"] Jul 14 14:07:48.462659 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:48.462337 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" podUID="8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" containerName="istio-proxy" containerID="cri-o://5bb78244f55bc98401d60a5a43ebceddad8661ce7501ef1d638d7bf087148b4e" gracePeriod=30 Jul 14 14:07:48.483107 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:48.483033 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:48.713226 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:48.713131 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:07:53.472018 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.471978 2583 patch_prober.go:28] interesting pod/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": EOF" start-of-body= Jul 14 14:07:53.472414 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.472051 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" podUID="8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": EOF" Jul 14 14:07:53.714971 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.714945 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:07:53.758693 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.758663 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-envoy\") pod \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " Jul 14 14:07:53.758893 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.758712 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-podinfo\") pod \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " Jul 14 14:07:53.758893 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.758738 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-data\") pod \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " Jul 14 14:07:53.758893 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.758774 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-credential-socket\") pod \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " Jul 14 14:07:53.758893 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.758870 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-workload-socket\") pod \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " Jul 14 14:07:53.759153 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.758901 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-workload-certs\") pod \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " Jul 14 14:07:53.759153 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.758940 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzvnt\" (UniqueName: \"kubernetes.io/projected/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-kube-api-access-fzvnt\") pod \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " Jul 14 14:07:53.759153 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.758974 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istiod-ca-cert\") pod \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " Jul 14 14:07:53.759153 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.758997 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-token\") pod \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\" (UID: \"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb\") " Jul 14 14:07:53.759357 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.759145 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" (UID: "8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:07:53.759434 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.759414 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-workload-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:07:53.759497 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.759412 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" (UID: "8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:07:53.759666 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.759639 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" (UID: "8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:07:53.759851 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.759828 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" (UID: "8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:07:53.760014 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.759983 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-data" (OuterVolumeSpecName: "istio-data") pod "8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" (UID: "8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:07:53.761283 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.761246 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-token" (OuterVolumeSpecName: "istio-token") pod "8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" (UID: "8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:07:53.761383 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.761314 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" (UID: "8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:07:53.761471 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.761453 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-kube-api-access-fzvnt" (OuterVolumeSpecName: "kube-api-access-fzvnt") pod "8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" (UID: "8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb"). InnerVolumeSpecName "kube-api-access-fzvnt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:07:53.761572 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.761554 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" (UID: "8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 14:07:53.860348 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.860313 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-workload-certs\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:07:53.860348 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.860348 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzvnt\" (UniqueName: \"kubernetes.io/projected/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-kube-api-access-fzvnt\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:07:53.860555 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.860363 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istiod-ca-cert\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:07:53.860555 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.860373 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-token\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:07:53.860555 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.860380 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-envoy\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:07:53.860555 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.860388 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-podinfo\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:07:53.860555 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.860395 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-istio-data\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:07:53.860555 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:53.860402 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb-credential-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:07:54.397673 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:54.397640 2583 generic.go:356] "Generic (PLEG): container finished" podID="8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" containerID="5bb78244f55bc98401d60a5a43ebceddad8661ce7501ef1d638d7bf087148b4e" exitCode=0 Jul 14 14:07:54.397852 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:54.397686 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" Jul 14 14:07:54.397852 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:54.397691 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" event={"ID":"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb","Type":"ContainerDied","Data":"5bb78244f55bc98401d60a5a43ebceddad8661ce7501ef1d638d7bf087148b4e"} Jul 14 14:07:54.397852 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:54.397805 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd" event={"ID":"8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb","Type":"ContainerDied","Data":"136f2d33fd1e9167ea7a3393be15ef844d5ea62dc1705b1877444a89c8c873b0"} Jul 14 14:07:54.397852 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:54.397829 2583 scope.go:117] "RemoveContainer" containerID="5bb78244f55bc98401d60a5a43ebceddad8661ce7501ef1d638d7bf087148b4e" Jul 14 14:07:54.407590 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:54.407570 2583 scope.go:117] "RemoveContainer" containerID="2dc141ccbd3a8c444db463093fb47d7888b5b467558676cfbae59eaf4daba246" Jul 14 14:07:54.415699 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:54.415679 2583 scope.go:117] "RemoveContainer" containerID="5bb78244f55bc98401d60a5a43ebceddad8661ce7501ef1d638d7bf087148b4e" Jul 14 14:07:54.415943 ip-10-0-142-94 kubenswrapper[2583]: E0714 14:07:54.415927 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bb78244f55bc98401d60a5a43ebceddad8661ce7501ef1d638d7bf087148b4e\": container with ID starting with 5bb78244f55bc98401d60a5a43ebceddad8661ce7501ef1d638d7bf087148b4e not found: ID does not exist" containerID="5bb78244f55bc98401d60a5a43ebceddad8661ce7501ef1d638d7bf087148b4e" Jul 14 14:07:54.415992 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:54.415951 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bb78244f55bc98401d60a5a43ebceddad8661ce7501ef1d638d7bf087148b4e"} err="failed to get container status \"5bb78244f55bc98401d60a5a43ebceddad8661ce7501ef1d638d7bf087148b4e\": rpc error: code = NotFound desc = could not find container \"5bb78244f55bc98401d60a5a43ebceddad8661ce7501ef1d638d7bf087148b4e\": container with ID starting with 5bb78244f55bc98401d60a5a43ebceddad8661ce7501ef1d638d7bf087148b4e not found: ID does not exist" Jul 14 14:07:54.415992 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:54.415967 2583 scope.go:117] "RemoveContainer" containerID="2dc141ccbd3a8c444db463093fb47d7888b5b467558676cfbae59eaf4daba246" Jul 14 14:07:54.416228 ip-10-0-142-94 kubenswrapper[2583]: E0714 14:07:54.416210 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dc141ccbd3a8c444db463093fb47d7888b5b467558676cfbae59eaf4daba246\": container with ID starting with 2dc141ccbd3a8c444db463093fb47d7888b5b467558676cfbae59eaf4daba246 not found: ID does not exist" containerID="2dc141ccbd3a8c444db463093fb47d7888b5b467558676cfbae59eaf4daba246" Jul 14 14:07:54.416273 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:54.416232 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dc141ccbd3a8c444db463093fb47d7888b5b467558676cfbae59eaf4daba246"} err="failed to get container status \"2dc141ccbd3a8c444db463093fb47d7888b5b467558676cfbae59eaf4daba246\": rpc error: code = NotFound desc = could not find container \"2dc141ccbd3a8c444db463093fb47d7888b5b467558676cfbae59eaf4daba246\": container with ID starting with 2dc141ccbd3a8c444db463093fb47d7888b5b467558676cfbae59eaf4daba246 not found: ID does not exist" Jul 14 14:07:54.421186 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:54.421166 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd"] Jul 14 14:07:54.425673 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:54.425654 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-f2a083-openshift-default-75cfc697cf-cmbqd"] Jul 14 14:07:54.493584 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:54.493550 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" path="/var/lib/kubelet/pods/8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb/volumes" Jul 14 14:07:59.298391 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:07:59.298344 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:03.939705 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:03.939672 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:10.229757 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:10.229719 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:10.331289 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:10.331255 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:14.632088 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:14.632037 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt"] Jul 14 14:08:14.632556 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:14.632314 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" podUID="1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" containerName="istio-proxy" containerID="cri-o://985a4740b84041ed8dca211adf3745d3a9f5d35e5c7026e0664596ad1d34ec11" gracePeriod=30 Jul 14 14:08:14.719699 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:14.719659 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:14.904276 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:14.904192 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:15.262241 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.262205 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:15.571019 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.570927 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw"] Jul 14 14:08:15.571560 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.571543 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" containerName="istio-proxy" Jul 14 14:08:15.571622 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.571564 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" containerName="istio-proxy" Jul 14 14:08:15.571622 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.571578 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" containerName="istio-proxy" Jul 14 14:08:15.571622 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.571587 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" containerName="istio-proxy" Jul 14 14:08:15.571729 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.571672 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" containerName="istio-proxy" Jul 14 14:08:15.571729 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.571684 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fb7b26b-17fd-4c7a-a0db-b004ae1d0cfb" containerName="istio-proxy" Jul 14 14:08:15.575338 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.575320 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.577924 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.577901 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-9c0e14-openshift-default-dockercfg-hrzn6\"" Jul 14 14:08:15.585742 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.585716 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw"] Jul 14 14:08:15.599626 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.599600 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:15.676021 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.675978 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-credential-socket\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.676548 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.676038 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-podinfo\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.676548 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.676111 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-workload-socket\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.676548 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.676159 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-data\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.676548 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.676211 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-envoy\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.676548 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.676267 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39ea6257-1b7a-4bfe-92be-f9a728892902-istiod-ca-cert\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.676548 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.676336 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-token\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.676548 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.676370 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4nb6\" (UniqueName: \"kubernetes.io/projected/39ea6257-1b7a-4bfe-92be-f9a728892902-kube-api-access-d4nb6\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.676548 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.676409 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-workload-certs\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.775871 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.775840 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:15.777036 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.777013 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-credential-socket\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.777177 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.777047 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-podinfo\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.777177 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.777114 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-workload-socket\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.777177 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.777142 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-data\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.777350 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.777185 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-envoy\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.777350 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.777313 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39ea6257-1b7a-4bfe-92be-f9a728892902-istiod-ca-cert\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.777464 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.777366 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-token\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.777464 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.777399 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4nb6\" (UniqueName: \"kubernetes.io/projected/39ea6257-1b7a-4bfe-92be-f9a728892902-kube-api-access-d4nb6\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.777464 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.777445 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-workload-certs\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.777753 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.777592 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-workload-socket\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.777753 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.777619 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-data\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.777753 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.777707 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-credential-socket\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.777965 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.777944 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-workload-certs\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.778175 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.778151 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39ea6257-1b7a-4bfe-92be-f9a728892902-istiod-ca-cert\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.779689 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.779670 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-envoy\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.779850 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.779831 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-podinfo\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.786214 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.786190 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-token\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.786310 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.786289 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4nb6\" (UniqueName: \"kubernetes.io/projected/39ea6257-1b7a-4bfe-92be-f9a728892902-kube-api-access-d4nb6\") pod \"e2e-shared-a-9c0e14-openshift-default-65946787f-47stw\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:15.887331 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:15.887221 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:16.030550 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:16.030522 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw"] Jul 14 14:08:16.033239 ip-10-0-142-94 kubenswrapper[2583]: W0714 14:08:16.033201 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39ea6257_1b7a_4bfe_92be_f9a728892902.slice/crio-914bcd8026c4ce20eee5500bc627afcf734b164c443b3ff38cda1ef2deff3ece WatchSource:0}: Error finding container 914bcd8026c4ce20eee5500bc627afcf734b164c443b3ff38cda1ef2deff3ece: Status 404 returned error can't find the container with id 914bcd8026c4ce20eee5500bc627afcf734b164c443b3ff38cda1ef2deff3ece Jul 14 14:08:16.035616 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:16.035583 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:08:16.035740 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:16.035665 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:08:16.035740 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:16.035694 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:08:16.511185 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:16.511150 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" event={"ID":"39ea6257-1b7a-4bfe-92be-f9a728892902","Type":"ContainerStarted","Data":"8476f26ff7a52e888e606e6c2d06d1fdbdd42e8e3e1a2bcfbe5bb053e418d0be"} Jul 14 14:08:16.511383 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:16.511212 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" event={"ID":"39ea6257-1b7a-4bfe-92be-f9a728892902","Type":"ContainerStarted","Data":"914bcd8026c4ce20eee5500bc627afcf734b164c443b3ff38cda1ef2deff3ece"} Jul 14 14:08:16.535818 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:16.535750 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" podStartSLOduration=1.535728605 podStartE2EDuration="1.535728605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:08:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:08:16.534550034 +0000 UTC m=+2460.061700671" watchObservedRunningTime="2026-07-14 14:08:16.535728605 +0000 UTC m=+2460.062879241" Jul 14 14:08:16.625700 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:16.625664 2583 patch_prober.go:28] interesting pod/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 14:08:16.625872 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:16.625734 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" podUID="1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 14:08:16.888263 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:16.888168 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:17.892932 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:17.892904 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:18.462682 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:18.462644 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:18.521723 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:18.521696 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:18.522859 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:18.522837 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:18.551200 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:18.551149 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:18.749704 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:18.749670 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:19.028529 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:19.028441 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:19.314608 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:19.314508 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:19.415573 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:19.415540 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:19.907727 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:19.907663 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:08:20.019029 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.018994 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-credential-socket\") pod \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " Jul 14 14:08:20.019263 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.019056 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istiod-ca-cert\") pod \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " Jul 14 14:08:20.019263 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.019098 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-workload-socket\") pod \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " Jul 14 14:08:20.019263 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.019152 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-px2rd\" (UniqueName: \"kubernetes.io/projected/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-kube-api-access-px2rd\") pod \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " Jul 14 14:08:20.019263 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.019179 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-podinfo\") pod \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " Jul 14 14:08:20.019263 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.019200 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-workload-certs\") pod \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " Jul 14 14:08:20.019263 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.019214 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-envoy\") pod \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " Jul 14 14:08:20.019263 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.019237 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-token\") pod \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " Jul 14 14:08:20.019651 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.019288 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-data\") pod \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\" (UID: \"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926\") " Jul 14 14:08:20.019651 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.019344 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" (UID: "1ef38dc3-c7bb-4d5c-b883-6d1aa227c926"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:08:20.019651 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.019440 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" (UID: "1ef38dc3-c7bb-4d5c-b883-6d1aa227c926"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:08:20.019651 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.019586 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" (UID: "1ef38dc3-c7bb-4d5c-b883-6d1aa227c926"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:08:20.019651 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.019620 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" (UID: "1ef38dc3-c7bb-4d5c-b883-6d1aa227c926"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:08:20.019651 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.019644 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-workload-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:08:20.019902 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.019663 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-credential-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:08:20.020393 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.020368 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-data" (OuterVolumeSpecName: "istio-data") pod "1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" (UID: "1ef38dc3-c7bb-4d5c-b883-6d1aa227c926"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:08:20.021500 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.021468 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" (UID: "1ef38dc3-c7bb-4d5c-b883-6d1aa227c926"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:08:20.021616 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.021577 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" (UID: "1ef38dc3-c7bb-4d5c-b883-6d1aa227c926"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 14:08:20.021846 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.021828 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-token" (OuterVolumeSpecName: "istio-token") pod "1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" (UID: "1ef38dc3-c7bb-4d5c-b883-6d1aa227c926"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:08:20.021905 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.021844 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-kube-api-access-px2rd" (OuterVolumeSpecName: "kube-api-access-px2rd") pod "1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" (UID: "1ef38dc3-c7bb-4d5c-b883-6d1aa227c926"). InnerVolumeSpecName "kube-api-access-px2rd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:08:20.120326 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.120289 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-px2rd\" (UniqueName: \"kubernetes.io/projected/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-kube-api-access-px2rd\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:08:20.120326 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.120319 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-podinfo\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:08:20.120326 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.120328 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-workload-certs\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:08:20.120326 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.120336 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-envoy\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:08:20.120802 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.120343 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-token\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:08:20.120802 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.120352 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istio-data\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:08:20.120802 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.120360 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926-istiod-ca-cert\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:08:20.535713 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.535673 2583 generic.go:356] "Generic (PLEG): container finished" podID="1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" containerID="985a4740b84041ed8dca211adf3745d3a9f5d35e5c7026e0664596ad1d34ec11" exitCode=0 Jul 14 14:08:20.535886 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.535729 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" Jul 14 14:08:20.535886 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.535750 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" event={"ID":"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926","Type":"ContainerDied","Data":"985a4740b84041ed8dca211adf3745d3a9f5d35e5c7026e0664596ad1d34ec11"} Jul 14 14:08:20.535886 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.535800 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt" event={"ID":"1ef38dc3-c7bb-4d5c-b883-6d1aa227c926","Type":"ContainerDied","Data":"a253adcad38c757d34b1cd8fa24c87204108579222b4cd4738d7eaa52ad96686"} Jul 14 14:08:20.535886 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.535821 2583 scope.go:117] "RemoveContainer" containerID="985a4740b84041ed8dca211adf3745d3a9f5d35e5c7026e0664596ad1d34ec11" Jul 14 14:08:20.544798 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.544781 2583 scope.go:117] "RemoveContainer" containerID="11e02b2de2faed002bd86b261089b36d9a1b7f7de2d5bf423ec398accad67184" Jul 14 14:08:20.552907 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.552888 2583 scope.go:117] "RemoveContainer" containerID="985a4740b84041ed8dca211adf3745d3a9f5d35e5c7026e0664596ad1d34ec11" Jul 14 14:08:20.553275 ip-10-0-142-94 kubenswrapper[2583]: E0714 14:08:20.553239 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"985a4740b84041ed8dca211adf3745d3a9f5d35e5c7026e0664596ad1d34ec11\": container with ID starting with 985a4740b84041ed8dca211adf3745d3a9f5d35e5c7026e0664596ad1d34ec11 not found: ID does not exist" containerID="985a4740b84041ed8dca211adf3745d3a9f5d35e5c7026e0664596ad1d34ec11" Jul 14 14:08:20.553398 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.553292 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"985a4740b84041ed8dca211adf3745d3a9f5d35e5c7026e0664596ad1d34ec11"} err="failed to get container status \"985a4740b84041ed8dca211adf3745d3a9f5d35e5c7026e0664596ad1d34ec11\": rpc error: code = NotFound desc = could not find container \"985a4740b84041ed8dca211adf3745d3a9f5d35e5c7026e0664596ad1d34ec11\": container with ID starting with 985a4740b84041ed8dca211adf3745d3a9f5d35e5c7026e0664596ad1d34ec11 not found: ID does not exist" Jul 14 14:08:20.553398 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.553319 2583 scope.go:117] "RemoveContainer" containerID="11e02b2de2faed002bd86b261089b36d9a1b7f7de2d5bf423ec398accad67184" Jul 14 14:08:20.553689 ip-10-0-142-94 kubenswrapper[2583]: E0714 14:08:20.553660 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11e02b2de2faed002bd86b261089b36d9a1b7f7de2d5bf423ec398accad67184\": container with ID starting with 11e02b2de2faed002bd86b261089b36d9a1b7f7de2d5bf423ec398accad67184 not found: ID does not exist" containerID="11e02b2de2faed002bd86b261089b36d9a1b7f7de2d5bf423ec398accad67184" Jul 14 14:08:20.553792 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.553697 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11e02b2de2faed002bd86b261089b36d9a1b7f7de2d5bf423ec398accad67184"} err="failed to get container status \"11e02b2de2faed002bd86b261089b36d9a1b7f7de2d5bf423ec398accad67184\": rpc error: code = NotFound desc = could not find container \"11e02b2de2faed002bd86b261089b36d9a1b7f7de2d5bf423ec398accad67184\": container with ID starting with 11e02b2de2faed002bd86b261089b36d9a1b7f7de2d5bf423ec398accad67184 not found: ID does not exist" Jul 14 14:08:20.555798 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.555777 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt"] Jul 14 14:08:20.560818 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:20.560793 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-1816dc-openshift-default-54b9bd75f9-m87vt"] Jul 14 14:08:22.493952 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:22.493914 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" path="/var/lib/kubelet/pods/1ef38dc3-c7bb-4d5c-b883-6d1aa227c926/volumes" Jul 14 14:08:23.011330 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.011296 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:23.349349 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.349263 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:23.647007 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.646921 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8"] Jul 14 14:08:23.647538 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.647517 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" containerName="istio-proxy" Jul 14 14:08:23.647620 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.647542 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" containerName="istio-proxy" Jul 14 14:08:23.647749 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.647727 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" containerName="istio-proxy" Jul 14 14:08:23.647842 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.647776 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" containerName="istio-proxy" Jul 14 14:08:23.649952 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.649932 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.652459 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.652435 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-ff9f03-openshift-default-dockercfg-x67k6\"" Jul 14 14:08:23.663533 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.663514 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8"] Jul 14 14:08:23.689158 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.689132 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:23.750007 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.749971 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-envoy\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.750192 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.750023 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-data\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.750192 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.750098 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-token\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.750192 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.750164 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcglx\" (UniqueName: \"kubernetes.io/projected/46003e01-fd2c-40c5-ae09-f1229ce68598-kube-api-access-gcglx\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.750360 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.750281 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-credential-socket\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.750360 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.750303 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-podinfo\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.750360 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.750323 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-workload-certs\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.750360 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.750343 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-workload-socket\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.750594 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.750375 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46003e01-fd2c-40c5-ae09-f1229ce68598-istiod-ca-cert\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.837863 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.837827 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:23.851071 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.851039 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-data\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.851236 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.851111 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-token\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.851236 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.851134 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcglx\" (UniqueName: \"kubernetes.io/projected/46003e01-fd2c-40c5-ae09-f1229ce68598-kube-api-access-gcglx\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.851236 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.851195 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-credential-socket\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.851236 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.851220 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-podinfo\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.851441 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.851243 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-workload-certs\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.851441 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.851267 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-workload-socket\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.851441 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.851295 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46003e01-fd2c-40c5-ae09-f1229ce68598-istiod-ca-cert\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.851441 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.851363 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-envoy\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.851652 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.851494 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-data\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.851652 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.851601 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-credential-socket\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.851732 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.851692 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-workload-socket\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.851789 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.851765 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-workload-certs\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.852092 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.852057 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46003e01-fd2c-40c5-ae09-f1229ce68598-istiod-ca-cert\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.853942 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.853919 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-envoy\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.854296 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.854272 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-podinfo\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.860153 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.860131 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcglx\" (UniqueName: \"kubernetes.io/projected/46003e01-fd2c-40c5-ae09-f1229ce68598-kube-api-access-gcglx\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.860265 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.860242 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-token\") pod \"e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:23.960788 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:23.960706 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:24.101898 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:24.101874 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8"] Jul 14 14:08:24.104713 ip-10-0-142-94 kubenswrapper[2583]: W0714 14:08:24.104683 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46003e01_fd2c_40c5_ae09_f1229ce68598.slice/crio-a63e1f463178a6840c98833f00b2cae5f6b2bf86af4eeb14e84a6af532bb76cb WatchSource:0}: Error finding container a63e1f463178a6840c98833f00b2cae5f6b2bf86af4eeb14e84a6af532bb76cb: Status 404 returned error can't find the container with id a63e1f463178a6840c98833f00b2cae5f6b2bf86af4eeb14e84a6af532bb76cb Jul 14 14:08:24.107024 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:24.106991 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:08:24.107153 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:24.107056 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:08:24.107153 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:24.107103 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:08:24.560139 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:24.560070 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" event={"ID":"46003e01-fd2c-40c5-ae09-f1229ce68598","Type":"ContainerStarted","Data":"ea8260286e499a43c74730212aea692aa36cbffdbcf47d4d29a300880e059ef8"} Jul 14 14:08:24.560335 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:24.560171 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" event={"ID":"46003e01-fd2c-40c5-ae09-f1229ce68598","Type":"ContainerStarted","Data":"a63e1f463178a6840c98833f00b2cae5f6b2bf86af4eeb14e84a6af532bb76cb"} Jul 14 14:08:24.581181 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:24.581114 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" podStartSLOduration=1.581094818 podStartE2EDuration="1.581094818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:08:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:08:24.580542373 +0000 UTC m=+2468.107693009" watchObservedRunningTime="2026-07-14 14:08:24.581094818 +0000 UTC m=+2468.108245447" Jul 14 14:08:24.966463 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:24.961345 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:25.966942 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:25.966906 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:26.509442 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:26.509399 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:26.571174 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:26.571139 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:26.572146 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:26.572117 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:26.597651 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:26.597604 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:26.850113 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:26.849953 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:27.034571 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:27.034530 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:27.342046 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:27.342000 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:27.440103 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:27.440049 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:31.035067 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:31.035033 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:31.598908 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:31.598857 2583 generic.go:356] "Generic (PLEG): container finished" podID="39ea6257-1b7a-4bfe-92be-f9a728892902" containerID="8476f26ff7a52e888e606e6c2d06d1fdbdd42e8e3e1a2bcfbe5bb053e418d0be" exitCode=0 Jul 14 14:08:31.599170 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:31.598924 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" event={"ID":"39ea6257-1b7a-4bfe-92be-f9a728892902","Type":"ContainerDied","Data":"8476f26ff7a52e888e606e6c2d06d1fdbdd42e8e3e1a2bcfbe5bb053e418d0be"} Jul 14 14:08:31.599374 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:31.599360 2583 scope.go:117] "RemoveContainer" containerID="8476f26ff7a52e888e606e6c2d06d1fdbdd42e8e3e1a2bcfbe5bb053e418d0be" Jul 14 14:08:31.600562 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:31.600528 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:08:31.600673 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:31.600604 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:08:31.600673 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:31.600636 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:08:32.607623 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:32.607587 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" event={"ID":"39ea6257-1b7a-4bfe-92be-f9a728892902","Type":"ContainerStarted","Data":"44159adfa6e9d16b6fe0a0eaaf70b8bf417edd5975c94dc213c0d5b728b39917"} Jul 14 14:08:32.887984 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:32.887888 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:32.888915 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:32.888891 2583 patch_prober.go:28] interesting pod/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 14 14:08:32.888973 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:32.888943 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" podUID="39ea6257-1b7a-4bfe-92be-f9a728892902" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 14 14:08:33.521633 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:33.521598 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:33.887638 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:33.887548 2583 patch_prober.go:28] interesting pod/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 14 14:08:33.887638 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:33.887602 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" podUID="39ea6257-1b7a-4bfe-92be-f9a728892902" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 14 14:08:34.888297 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:34.888257 2583 patch_prober.go:28] interesting pod/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 14 14:08:34.888702 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:34.888316 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" podUID="39ea6257-1b7a-4bfe-92be-f9a728892902" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 14 14:08:35.888638 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:35.888600 2583 patch_prober.go:28] interesting pod/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 14 14:08:35.889043 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:35.888673 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" podUID="39ea6257-1b7a-4bfe-92be-f9a728892902" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 14 14:08:36.892503 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:36.892466 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:36.893833 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:36.893808 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:08:39.648701 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:39.648666 2583 generic.go:356] "Generic (PLEG): container finished" podID="46003e01-fd2c-40c5-ae09-f1229ce68598" containerID="ea8260286e499a43c74730212aea692aa36cbffdbcf47d4d29a300880e059ef8" exitCode=0 Jul 14 14:08:39.648701 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:39.648705 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" event={"ID":"46003e01-fd2c-40c5-ae09-f1229ce68598","Type":"ContainerDied","Data":"ea8260286e499a43c74730212aea692aa36cbffdbcf47d4d29a300880e059ef8"} Jul 14 14:08:39.649266 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:39.649101 2583 scope.go:117] "RemoveContainer" containerID="ea8260286e499a43c74730212aea692aa36cbffdbcf47d4d29a300880e059ef8" Jul 14 14:08:39.650138 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:39.650107 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:08:39.650248 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:39.650183 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:08:39.650248 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:39.650213 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 14:08:40.100609 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:40.100579 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:40.201733 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:40.201692 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:40.657586 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:40.657551 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" event={"ID":"46003e01-fd2c-40c5-ae09-f1229ce68598","Type":"ContainerStarted","Data":"572783d5dda31e2c38c784f929b83f0f3b0105910a850bb38c758aea28410253"} Jul 14 14:08:40.961152 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:40.961046 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:40.962492 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:40.962466 2583 patch_prober.go:28] interesting pod/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 14 14:08:40.962624 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:40.962531 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" podUID="46003e01-fd2c-40c5-ae09-f1229ce68598" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 14 14:08:41.571436 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:41.571386 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:41.961476 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:41.961388 2583 patch_prober.go:28] interesting pod/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 14 14:08:41.961476 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:41.961443 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" podUID="46003e01-fd2c-40c5-ae09-f1229ce68598" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 14 14:08:42.961208 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:42.961168 2583 patch_prober.go:28] interesting pod/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 14 14:08:42.961399 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:42.961242 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" podUID="46003e01-fd2c-40c5-ae09-f1229ce68598" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 14 14:08:43.961518 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:43.961477 2583 patch_prober.go:28] interesting pod/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 14 14:08:43.961910 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:43.961537 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" podUID="46003e01-fd2c-40c5-ae09-f1229ce68598" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 14 14:08:44.965156 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:44.965126 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:44.966013 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:44.965994 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:08:49.270771 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:49.270737 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:49.393552 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:49.393511 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:08:57.093302 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:08:57.093264 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:09:21.971875 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:09:21.971816 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:09:22.101789 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:09:22.101756 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:09:22.630700 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:09:22.630642 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:10:43.682771 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:10:43.682677 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:10:45.235456 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:10:45.235414 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:11:00.623529 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:00.623490 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:11:28.819628 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:28.819591 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:11:29.729775 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:29.729712 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:11:38.793378 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.793341 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv"] Jul 14 14:11:38.793970 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.793948 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" containerName="istio-proxy" Jul 14 14:11:38.794013 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.793976 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ef38dc3-c7bb-4d5c-b883-6d1aa227c926" containerName="istio-proxy" Jul 14 14:11:38.797431 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.797410 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:38.800341 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.800318 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9c0e14\"/\"default-dockercfg-6lvpd\"" Jul 14 14:11:38.801776 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.801759 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9c0e14\"/\"kube-root-ca.crt\"" Jul 14 14:11:38.801897 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.801836 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9c0e14\"/\"openshift-service-ca.crt\"" Jul 14 14:11:38.801951 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.801932 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9c0e14\"/\"sub-test-model-9c0e14-kserve-self-signed-certs\"" Jul 14 14:11:38.806747 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.806724 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv"] Jul 14 14:11:38.818526 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.818491 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:11:38.857974 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.857940 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-home\") pod \"sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:38.858196 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.857981 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-tmp-dir\") pod \"sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:38.858196 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.858003 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxkqm\" (UniqueName: \"kubernetes.io/projected/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-kube-api-access-fxkqm\") pod \"sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:38.858196 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.858138 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-tls-certs\") pod \"sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:38.858364 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.858227 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-model-cache\") pod \"sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:38.858364 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.858263 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-dshm\") pod \"sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:38.959563 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.959529 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-model-cache\") pod \"sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:38.959563 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.959569 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-dshm\") pod \"sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:38.959832 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.959602 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-home\") pod \"sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:38.959832 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.959623 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-tmp-dir\") pod \"sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:38.959832 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.959641 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxkqm\" (UniqueName: \"kubernetes.io/projected/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-kube-api-access-fxkqm\") pod \"sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:38.959832 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.959694 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-tls-certs\") pod \"sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:38.960062 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.960040 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-model-cache\") pod \"sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:38.960146 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.960124 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-home\") pod \"sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:38.960204 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.960183 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-tmp-dir\") pod \"sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:38.961899 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.961882 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-dshm\") pod \"sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:38.962330 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.962309 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-tls-certs\") pod \"sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:38.968797 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:38.968768 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxkqm\" (UniqueName: \"kubernetes.io/projected/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-kube-api-access-fxkqm\") pod \"sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:39.107515 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:39.107426 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:39.251562 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:39.251533 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv"] Jul 14 14:11:39.253325 ip-10-0-142-94 kubenswrapper[2583]: W0714 14:11:39.253293 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode56e0ce0_8f2b_4556_8ee2_a1609e15b8ca.slice/crio-f015d2b4c5bd671a52fb227cba99095a7734b32f4a9991b15adee451c5882987 WatchSource:0}: Error finding container f015d2b4c5bd671a52fb227cba99095a7734b32f4a9991b15adee451c5882987: Status 404 returned error can't find the container with id f015d2b4c5bd671a52fb227cba99095a7734b32f4a9991b15adee451c5882987 Jul 14 14:11:39.561619 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:39.561578 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" event={"ID":"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca","Type":"ContainerStarted","Data":"11e4049491a092d26574ba4305c0790a056e80945b62ee9d20729dfa4ee300e9"} Jul 14 14:11:39.561776 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:39.561630 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" event={"ID":"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca","Type":"ContainerStarted","Data":"f015d2b4c5bd671a52fb227cba99095a7734b32f4a9991b15adee451c5882987"} Jul 14 14:11:39.581612 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:39.581550 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" podStartSLOduration=1.581529974 podStartE2EDuration="1.581529974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:11:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:11:39.579335495 +0000 UTC m=+2663.106486135" watchObservedRunningTime="2026-07-14 14:11:39.581529974 +0000 UTC m=+2663.108680610" Jul 14 14:11:49.107562 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:49.107524 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:49.108030 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:49.107579 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:49.121126 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:49.121099 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:11:49.625482 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:11:49.625449 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:12:01.035707 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:01.035668 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:12:11.815600 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:11.815502 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:12:18.589584 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:18.589555 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 14:12:18.593130 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:18.593106 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lpnff_694177f0-32e8-4724-b626-ca46d6c476b5/ovn-acl-logging/0.log" Jul 14 14:12:37.497136 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:37.497068 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv"] Jul 14 14:12:37.516668 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:37.516629 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:12:38.621734 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:38.621693 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:12:51.700771 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:51.700733 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:12:52.518481 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.518416 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" podUID="e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca" containerName="main" containerID="cri-o://11e4049491a092d26574ba4305c0790a056e80945b62ee9d20729dfa4ee300e9" gracePeriod=45 Jul 14 14:12:52.772509 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.772444 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:12:52.794504 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.794471 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-model-cache\") pod \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " Jul 14 14:12:52.794670 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.794544 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-dshm\") pod \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " Jul 14 14:12:52.794670 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.794580 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-tls-certs\") pod \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " Jul 14 14:12:52.794670 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.794609 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-tmp-dir\") pod \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " Jul 14 14:12:52.794670 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.794644 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxkqm\" (UniqueName: \"kubernetes.io/projected/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-kube-api-access-fxkqm\") pod \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " Jul 14 14:12:52.794813 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.794689 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-home\") pod \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\" (UID: \"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca\") " Jul 14 14:12:52.795143 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.795101 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-home" (OuterVolumeSpecName: "home") pod "e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca" (UID: "e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:12:52.795143 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.795123 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca" (UID: "e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:12:52.795705 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.795672 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-model-cache" (OuterVolumeSpecName: "model-cache") pod "e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca" (UID: "e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:12:52.797211 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.797173 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-kube-api-access-fxkqm" (OuterVolumeSpecName: "kube-api-access-fxkqm") pod "e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca" (UID: "e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca"). InnerVolumeSpecName "kube-api-access-fxkqm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:12:52.797320 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.797220 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca" (UID: "e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 14:12:52.797320 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.797220 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-dshm" (OuterVolumeSpecName: "dshm") pod "e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca" (UID: "e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:12:52.895350 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.895311 2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-home\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:12:52.895350 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.895344 2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-model-cache\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:12:52.895350 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.895357 2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-dshm\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:12:52.895600 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.895370 2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-tls-certs\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:12:52.895600 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.895381 2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-tmp-dir\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:12:52.895600 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.895393 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxkqm\" (UniqueName: \"kubernetes.io/projected/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca-kube-api-access-fxkqm\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:12:52.938165 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.938127 2583 generic.go:356] "Generic (PLEG): container finished" podID="e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca" containerID="11e4049491a092d26574ba4305c0790a056e80945b62ee9d20729dfa4ee300e9" exitCode=0 Jul 14 14:12:52.938165 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.938169 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" Jul 14 14:12:52.938391 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.938193 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" event={"ID":"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca","Type":"ContainerDied","Data":"11e4049491a092d26574ba4305c0790a056e80945b62ee9d20729dfa4ee300e9"} Jul 14 14:12:52.938391 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.938224 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv" event={"ID":"e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca","Type":"ContainerDied","Data":"f015d2b4c5bd671a52fb227cba99095a7734b32f4a9991b15adee451c5882987"} Jul 14 14:12:52.938391 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.938240 2583 scope.go:117] "RemoveContainer" containerID="11e4049491a092d26574ba4305c0790a056e80945b62ee9d20729dfa4ee300e9" Jul 14 14:12:52.949074 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.949051 2583 scope.go:117] "RemoveContainer" containerID="11e4049491a092d26574ba4305c0790a056e80945b62ee9d20729dfa4ee300e9" Jul 14 14:12:52.949481 ip-10-0-142-94 kubenswrapper[2583]: E0714 14:12:52.949461 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11e4049491a092d26574ba4305c0790a056e80945b62ee9d20729dfa4ee300e9\": container with ID starting with 11e4049491a092d26574ba4305c0790a056e80945b62ee9d20729dfa4ee300e9 not found: ID does not exist" containerID="11e4049491a092d26574ba4305c0790a056e80945b62ee9d20729dfa4ee300e9" Jul 14 14:12:52.949545 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.949490 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11e4049491a092d26574ba4305c0790a056e80945b62ee9d20729dfa4ee300e9"} err="failed to get container status \"11e4049491a092d26574ba4305c0790a056e80945b62ee9d20729dfa4ee300e9\": rpc error: code = NotFound desc = could not find container \"11e4049491a092d26574ba4305c0790a056e80945b62ee9d20729dfa4ee300e9\": container with ID starting with 11e4049491a092d26574ba4305c0790a056e80945b62ee9d20729dfa4ee300e9 not found: ID does not exist" Jul 14 14:12:52.963119 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.963072 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv"] Jul 14 14:12:52.968906 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:52.968877 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-9c0e14/sub-test-model-9c0e14-kserve-5f5ff95d8c-7x4gv"] Jul 14 14:12:54.492971 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:12:54.492936 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca" path="/var/lib/kubelet/pods/e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca/volumes" Jul 14 14:13:09.422772 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:13:09.422734 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:13:41.014678 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:13:41.014634 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:13:41.116270 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:13:41.116234 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:13:46.870267 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:13:46.870219 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:13:46.972810 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:13:46.972780 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:13:50.084067 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:13:50.084035 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:13:50.184212 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:13:50.184174 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:13:54.208704 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:13:54.208660 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:13:55.132468 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:13:55.132431 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:14:15.929852 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:15.929810 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:14:16.033071 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:16.033024 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:14:25.926168 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:25.926121 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:14:30.771792 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:30.771752 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:14:37.224406 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:37.224374 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:14:37.304466 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:37.304423 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:14:41.201026 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:41.200985 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw"] Jul 14 14:14:41.201518 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:41.201248 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" podUID="39ea6257-1b7a-4bfe-92be-f9a728892902" containerName="istio-proxy" containerID="cri-o://44159adfa6e9d16b6fe0a0eaaf70b8bf417edd5975c94dc213c0d5b728b39917" gracePeriod=30 Jul 14 14:14:41.219514 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:41.219466 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:14:41.468357 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:41.468264 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:14:42.134515 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:42.134452 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:14:42.236797 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:42.236760 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:14:46.465319 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.465245 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:14:46.519935 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.519902 2583 generic.go:356] "Generic (PLEG): container finished" podID="39ea6257-1b7a-4bfe-92be-f9a728892902" containerID="44159adfa6e9d16b6fe0a0eaaf70b8bf417edd5975c94dc213c0d5b728b39917" exitCode=0 Jul 14 14:14:46.520148 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.519968 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" Jul 14 14:14:46.520148 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.519976 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" event={"ID":"39ea6257-1b7a-4bfe-92be-f9a728892902","Type":"ContainerDied","Data":"44159adfa6e9d16b6fe0a0eaaf70b8bf417edd5975c94dc213c0d5b728b39917"} Jul 14 14:14:46.520148 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.520028 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw" event={"ID":"39ea6257-1b7a-4bfe-92be-f9a728892902","Type":"ContainerDied","Data":"914bcd8026c4ce20eee5500bc627afcf734b164c443b3ff38cda1ef2deff3ece"} Jul 14 14:14:46.520148 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.520052 2583 scope.go:117] "RemoveContainer" containerID="44159adfa6e9d16b6fe0a0eaaf70b8bf417edd5975c94dc213c0d5b728b39917" Jul 14 14:14:46.530471 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.530447 2583 scope.go:117] "RemoveContainer" containerID="8476f26ff7a52e888e606e6c2d06d1fdbdd42e8e3e1a2bcfbe5bb053e418d0be" Jul 14 14:14:46.538429 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.538411 2583 scope.go:117] "RemoveContainer" containerID="44159adfa6e9d16b6fe0a0eaaf70b8bf417edd5975c94dc213c0d5b728b39917" Jul 14 14:14:46.538776 ip-10-0-142-94 kubenswrapper[2583]: E0714 14:14:46.538753 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44159adfa6e9d16b6fe0a0eaaf70b8bf417edd5975c94dc213c0d5b728b39917\": container with ID starting with 44159adfa6e9d16b6fe0a0eaaf70b8bf417edd5975c94dc213c0d5b728b39917 not found: ID does not exist" containerID="44159adfa6e9d16b6fe0a0eaaf70b8bf417edd5975c94dc213c0d5b728b39917" Jul 14 14:14:46.538841 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.538785 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44159adfa6e9d16b6fe0a0eaaf70b8bf417edd5975c94dc213c0d5b728b39917"} err="failed to get container status \"44159adfa6e9d16b6fe0a0eaaf70b8bf417edd5975c94dc213c0d5b728b39917\": rpc error: code = NotFound desc = could not find container \"44159adfa6e9d16b6fe0a0eaaf70b8bf417edd5975c94dc213c0d5b728b39917\": container with ID starting with 44159adfa6e9d16b6fe0a0eaaf70b8bf417edd5975c94dc213c0d5b728b39917 not found: ID does not exist" Jul 14 14:14:46.538841 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.538804 2583 scope.go:117] "RemoveContainer" containerID="8476f26ff7a52e888e606e6c2d06d1fdbdd42e8e3e1a2bcfbe5bb053e418d0be" Jul 14 14:14:46.539060 ip-10-0-142-94 kubenswrapper[2583]: E0714 14:14:46.539045 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8476f26ff7a52e888e606e6c2d06d1fdbdd42e8e3e1a2bcfbe5bb053e418d0be\": container with ID starting with 8476f26ff7a52e888e606e6c2d06d1fdbdd42e8e3e1a2bcfbe5bb053e418d0be not found: ID does not exist" containerID="8476f26ff7a52e888e606e6c2d06d1fdbdd42e8e3e1a2bcfbe5bb053e418d0be" Jul 14 14:14:46.539137 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.539064 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8476f26ff7a52e888e606e6c2d06d1fdbdd42e8e3e1a2bcfbe5bb053e418d0be"} err="failed to get container status \"8476f26ff7a52e888e606e6c2d06d1fdbdd42e8e3e1a2bcfbe5bb053e418d0be\": rpc error: code = NotFound desc = could not find container \"8476f26ff7a52e888e606e6c2d06d1fdbdd42e8e3e1a2bcfbe5bb053e418d0be\": container with ID starting with 8476f26ff7a52e888e606e6c2d06d1fdbdd42e8e3e1a2bcfbe5bb053e418d0be not found: ID does not exist" Jul 14 14:14:46.600171 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.600132 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-podinfo\") pod \"39ea6257-1b7a-4bfe-92be-f9a728892902\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " Jul 14 14:14:46.600358 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.600189 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-workload-socket\") pod \"39ea6257-1b7a-4bfe-92be-f9a728892902\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " Jul 14 14:14:46.600358 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.600227 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-data\") pod \"39ea6257-1b7a-4bfe-92be-f9a728892902\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " Jul 14 14:14:46.600358 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.600249 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-token\") pod \"39ea6257-1b7a-4bfe-92be-f9a728892902\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " Jul 14 14:14:46.600358 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.600284 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-envoy\") pod \"39ea6257-1b7a-4bfe-92be-f9a728892902\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " Jul 14 14:14:46.600358 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.600354 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39ea6257-1b7a-4bfe-92be-f9a728892902-istiod-ca-cert\") pod \"39ea6257-1b7a-4bfe-92be-f9a728892902\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " Jul 14 14:14:46.600610 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.600378 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4nb6\" (UniqueName: \"kubernetes.io/projected/39ea6257-1b7a-4bfe-92be-f9a728892902-kube-api-access-d4nb6\") pod \"39ea6257-1b7a-4bfe-92be-f9a728892902\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " Jul 14 14:14:46.600610 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.600421 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-workload-certs\") pod \"39ea6257-1b7a-4bfe-92be-f9a728892902\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " Jul 14 14:14:46.600610 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.600567 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "39ea6257-1b7a-4bfe-92be-f9a728892902" (UID: "39ea6257-1b7a-4bfe-92be-f9a728892902"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:14:46.600768 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.600638 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-credential-socket\") pod \"39ea6257-1b7a-4bfe-92be-f9a728892902\" (UID: \"39ea6257-1b7a-4bfe-92be-f9a728892902\") " Jul 14 14:14:46.600768 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.600692 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "39ea6257-1b7a-4bfe-92be-f9a728892902" (UID: "39ea6257-1b7a-4bfe-92be-f9a728892902"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:14:46.602297 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.600952 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "39ea6257-1b7a-4bfe-92be-f9a728892902" (UID: "39ea6257-1b7a-4bfe-92be-f9a728892902"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:14:46.602297 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.600997 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39ea6257-1b7a-4bfe-92be-f9a728892902-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "39ea6257-1b7a-4bfe-92be-f9a728892902" (UID: "39ea6257-1b7a-4bfe-92be-f9a728892902"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:14:46.602297 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.601177 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-data" (OuterVolumeSpecName: "istio-data") pod "39ea6257-1b7a-4bfe-92be-f9a728892902" (UID: "39ea6257-1b7a-4bfe-92be-f9a728892902"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:14:46.602297 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.601506 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39ea6257-1b7a-4bfe-92be-f9a728892902-istiod-ca-cert\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:14:46.602297 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.601526 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-workload-certs\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:14:46.602297 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.601541 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-credential-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:14:46.602297 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.601554 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-workload-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:14:46.602297 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.601568 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-data\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:14:46.603063 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.603041 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-token" (OuterVolumeSpecName: "istio-token") pod "39ea6257-1b7a-4bfe-92be-f9a728892902" (UID: "39ea6257-1b7a-4bfe-92be-f9a728892902"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:14:46.603063 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.603053 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "39ea6257-1b7a-4bfe-92be-f9a728892902" (UID: "39ea6257-1b7a-4bfe-92be-f9a728892902"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 14:14:46.603208 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.603145 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "39ea6257-1b7a-4bfe-92be-f9a728892902" (UID: "39ea6257-1b7a-4bfe-92be-f9a728892902"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:14:46.604019 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.603993 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39ea6257-1b7a-4bfe-92be-f9a728892902-kube-api-access-d4nb6" (OuterVolumeSpecName: "kube-api-access-d4nb6") pod "39ea6257-1b7a-4bfe-92be-f9a728892902" (UID: "39ea6257-1b7a-4bfe-92be-f9a728892902"). InnerVolumeSpecName "kube-api-access-d4nb6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:14:46.702912 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.702879 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-podinfo\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:14:46.702912 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.702909 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-token\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:14:46.702912 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.702918 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39ea6257-1b7a-4bfe-92be-f9a728892902-istio-envoy\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:14:46.703241 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.702930 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4nb6\" (UniqueName: \"kubernetes.io/projected/39ea6257-1b7a-4bfe-92be-f9a728892902-kube-api-access-d4nb6\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:14:46.845069 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.845033 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw"] Jul 14 14:14:46.850888 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:46.850838 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-9c0e14-openshift-default-65946787f-47stw"] Jul 14 14:14:48.493729 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:48.493693 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39ea6257-1b7a-4bfe-92be-f9a728892902" path="/var/lib/kubelet/pods/39ea6257-1b7a-4bfe-92be-f9a728892902/volumes" Jul 14 14:14:51.991312 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:51.991278 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:14:56.674304 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:14:56.674266 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:15:03.511624 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:03.511589 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:15:03.612526 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:03.612489 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:15:07.222250 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:07.222208 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:15:07.327452 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:07.327416 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:15:07.359229 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:07.359194 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8"] Jul 14 14:15:07.359421 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:07.359396 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" podUID="46003e01-fd2c-40c5-ae09-f1229ce68598" containerName="istio-proxy" containerID="cri-o://572783d5dda31e2c38c784f929b83f0f3b0105910a850bb38c758aea28410253" gracePeriod=30 Jul 14 14:15:07.392296 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:07.392262 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:15:07.648909 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:07.648876 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:15:12.617993 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.617968 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:15:12.657250 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.657157 2583 generic.go:356] "Generic (PLEG): container finished" podID="46003e01-fd2c-40c5-ae09-f1229ce68598" containerID="572783d5dda31e2c38c784f929b83f0f3b0105910a850bb38c758aea28410253" exitCode=0 Jul 14 14:15:12.657250 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.657204 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" Jul 14 14:15:12.657250 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.657218 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" event={"ID":"46003e01-fd2c-40c5-ae09-f1229ce68598","Type":"ContainerDied","Data":"572783d5dda31e2c38c784f929b83f0f3b0105910a850bb38c758aea28410253"} Jul 14 14:15:12.657534 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.657267 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8" event={"ID":"46003e01-fd2c-40c5-ae09-f1229ce68598","Type":"ContainerDied","Data":"a63e1f463178a6840c98833f00b2cae5f6b2bf86af4eeb14e84a6af532bb76cb"} Jul 14 14:15:12.657534 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.657298 2583 scope.go:117] "RemoveContainer" containerID="572783d5dda31e2c38c784f929b83f0f3b0105910a850bb38c758aea28410253" Jul 14 14:15:12.665738 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.665712 2583 scope.go:117] "RemoveContainer" containerID="ea8260286e499a43c74730212aea692aa36cbffdbcf47d4d29a300880e059ef8" Jul 14 14:15:12.673574 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.673551 2583 scope.go:117] "RemoveContainer" containerID="572783d5dda31e2c38c784f929b83f0f3b0105910a850bb38c758aea28410253" Jul 14 14:15:12.673817 ip-10-0-142-94 kubenswrapper[2583]: E0714 14:15:12.673798 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"572783d5dda31e2c38c784f929b83f0f3b0105910a850bb38c758aea28410253\": container with ID starting with 572783d5dda31e2c38c784f929b83f0f3b0105910a850bb38c758aea28410253 not found: ID does not exist" containerID="572783d5dda31e2c38c784f929b83f0f3b0105910a850bb38c758aea28410253" Jul 14 14:15:12.673875 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.673831 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"572783d5dda31e2c38c784f929b83f0f3b0105910a850bb38c758aea28410253"} err="failed to get container status \"572783d5dda31e2c38c784f929b83f0f3b0105910a850bb38c758aea28410253\": rpc error: code = NotFound desc = could not find container \"572783d5dda31e2c38c784f929b83f0f3b0105910a850bb38c758aea28410253\": container with ID starting with 572783d5dda31e2c38c784f929b83f0f3b0105910a850bb38c758aea28410253 not found: ID does not exist" Jul 14 14:15:12.673875 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.673852 2583 scope.go:117] "RemoveContainer" containerID="ea8260286e499a43c74730212aea692aa36cbffdbcf47d4d29a300880e059ef8" Jul 14 14:15:12.674093 ip-10-0-142-94 kubenswrapper[2583]: E0714 14:15:12.674063 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea8260286e499a43c74730212aea692aa36cbffdbcf47d4d29a300880e059ef8\": container with ID starting with ea8260286e499a43c74730212aea692aa36cbffdbcf47d4d29a300880e059ef8 not found: ID does not exist" containerID="ea8260286e499a43c74730212aea692aa36cbffdbcf47d4d29a300880e059ef8" Jul 14 14:15:12.674172 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.674115 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea8260286e499a43c74730212aea692aa36cbffdbcf47d4d29a300880e059ef8"} err="failed to get container status \"ea8260286e499a43c74730212aea692aa36cbffdbcf47d4d29a300880e059ef8\": rpc error: code = NotFound desc = could not find container \"ea8260286e499a43c74730212aea692aa36cbffdbcf47d4d29a300880e059ef8\": container with ID starting with ea8260286e499a43c74730212aea692aa36cbffdbcf47d4d29a300880e059ef8 not found: ID does not exist" Jul 14 14:15:12.740961 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.740928 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46003e01-fd2c-40c5-ae09-f1229ce68598-istiod-ca-cert\") pod \"46003e01-fd2c-40c5-ae09-f1229ce68598\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " Jul 14 14:15:12.741172 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.740991 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-workload-socket\") pod \"46003e01-fd2c-40c5-ae09-f1229ce68598\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " Jul 14 14:15:12.741172 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.741012 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-token\") pod \"46003e01-fd2c-40c5-ae09-f1229ce68598\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " Jul 14 14:15:12.741172 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.741047 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-credential-socket\") pod \"46003e01-fd2c-40c5-ae09-f1229ce68598\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " Jul 14 14:15:12.741172 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.741076 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-podinfo\") pod \"46003e01-fd2c-40c5-ae09-f1229ce68598\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " Jul 14 14:15:12.741172 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.741148 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-envoy\") pod \"46003e01-fd2c-40c5-ae09-f1229ce68598\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " Jul 14 14:15:12.741490 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.741187 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-data\") pod \"46003e01-fd2c-40c5-ae09-f1229ce68598\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " Jul 14 14:15:12.741490 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.741213 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcglx\" (UniqueName: \"kubernetes.io/projected/46003e01-fd2c-40c5-ae09-f1229ce68598-kube-api-access-gcglx\") pod \"46003e01-fd2c-40c5-ae09-f1229ce68598\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " Jul 14 14:15:12.741490 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.741266 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-workload-certs\") pod \"46003e01-fd2c-40c5-ae09-f1229ce68598\" (UID: \"46003e01-fd2c-40c5-ae09-f1229ce68598\") " Jul 14 14:15:12.741490 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.741378 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "46003e01-fd2c-40c5-ae09-f1229ce68598" (UID: "46003e01-fd2c-40c5-ae09-f1229ce68598"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:15:12.741700 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.741428 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46003e01-fd2c-40c5-ae09-f1229ce68598-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "46003e01-fd2c-40c5-ae09-f1229ce68598" (UID: "46003e01-fd2c-40c5-ae09-f1229ce68598"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:15:12.741700 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.741648 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-workload-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:15:12.741700 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.741669 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46003e01-fd2c-40c5-ae09-f1229ce68598-istiod-ca-cert\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:15:12.742096 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.741734 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "46003e01-fd2c-40c5-ae09-f1229ce68598" (UID: "46003e01-fd2c-40c5-ae09-f1229ce68598"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:15:12.742096 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.741851 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "46003e01-fd2c-40c5-ae09-f1229ce68598" (UID: "46003e01-fd2c-40c5-ae09-f1229ce68598"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:15:12.742224 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.742177 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-data" (OuterVolumeSpecName: "istio-data") pod "46003e01-fd2c-40c5-ae09-f1229ce68598" (UID: "46003e01-fd2c-40c5-ae09-f1229ce68598"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:15:12.743385 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.743347 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "46003e01-fd2c-40c5-ae09-f1229ce68598" (UID: "46003e01-fd2c-40c5-ae09-f1229ce68598"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:15:12.743497 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.743407 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "46003e01-fd2c-40c5-ae09-f1229ce68598" (UID: "46003e01-fd2c-40c5-ae09-f1229ce68598"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 14:15:12.743497 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.743433 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-token" (OuterVolumeSpecName: "istio-token") pod "46003e01-fd2c-40c5-ae09-f1229ce68598" (UID: "46003e01-fd2c-40c5-ae09-f1229ce68598"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:15:12.743603 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.743564 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46003e01-fd2c-40c5-ae09-f1229ce68598-kube-api-access-gcglx" (OuterVolumeSpecName: "kube-api-access-gcglx") pod "46003e01-fd2c-40c5-ae09-f1229ce68598" (UID: "46003e01-fd2c-40c5-ae09-f1229ce68598"). InnerVolumeSpecName "kube-api-access-gcglx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:15:12.842509 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.842471 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-workload-certs\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:15:12.842509 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.842503 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-token\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:15:12.842509 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.842512 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-credential-socket\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:15:12.842754 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.842526 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-podinfo\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:15:12.842754 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.842549 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-envoy\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:15:12.842754 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.842557 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46003e01-fd2c-40c5-ae09-f1229ce68598-istio-data\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:15:12.842754 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.842565 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcglx\" (UniqueName: \"kubernetes.io/projected/46003e01-fd2c-40c5-ae09-f1229ce68598-kube-api-access-gcglx\") on node \"ip-10-0-142-94.ec2.internal\" DevicePath \"\"" Jul 14 14:15:12.983403 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.983368 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8"] Jul 14 14:15:12.987811 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:12.987785 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-ff9f03-openshift-default-74f8c4d4d7-r6dg8"] Jul 14 14:15:13.136842 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:13.136814 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-qx7mp_109d56a5-7738-4b1f-ab5b-410e96a8aafb/manager/0.log" Jul 14 14:15:13.272521 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:13.272490 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-48fz7_18984890-7e5d-4556-aadc-1cc54a152630/manager/0.log" Jul 14 14:15:13.406420 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:13.406385 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-78bfdb59d-gtpx2_6aadcd44-2422-4e08-9bb0-cee729bd6011/manager/0.log" Jul 14 14:15:13.521102 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:13.521057 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gtnwl_0b8ca6b7-4548-463a-a764-cbd5f9a5dbdb/server/0.log" Jul 14 14:15:13.635955 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:13.635878 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-jx5df_218fb2da-1c3e-4b3b-984a-0861060202a3/manager/0.log" Jul 14 14:15:14.032266 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:14.032244 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9464f545f-xqxn6_20c64d8b-5726-4930-8be6-02866c1b10ca/copy-manifests/0.log" Jul 14 14:15:14.039281 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:14.039254 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9464f545f-xqxn6_20c64d8b-5726-4930-8be6-02866c1b10ca/manager/0.log" Jul 14 14:15:14.493217 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:14.493137 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46003e01-fd2c-40c5-ae09-f1229ce68598" path="/var/lib/kubelet/pods/46003e01-fd2c-40c5-ae09-f1229ce68598/volumes" Jul 14 14:15:16.341049 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:16.341022 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-62xhp_b8741364-7e73-47a4-abd2-4fa8bc1b07cd/postgres/0.log" Jul 14 14:15:17.850135 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:17.850105 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-s7n6g_ec2f03ee-0cfb-47d2-92d6-04525e59aa26/manager/0.log" Jul 14 14:15:17.978193 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:17.978149 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-2nnxq_cbfcc3c6-4c72-40eb-a830-0b6b01c6947c/manager/0.log" Jul 14 14:15:18.229383 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:18.229275 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-67t8q_318072db-4ca4-4cd5-9b92-139d2abc7414/registry-server/0.log" Jul 14 14:15:18.358766 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:18.358730 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-pqbhm_6dea9fa8-c924-457d-bc4e-7477e5e8ff9f/manager/0.log" Jul 14 14:15:18.616636 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:18.616610 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-78tsv_2fa7beda-73cd-4e50-9a1d-0112b430a57c/manager/0.log" Jul 14 14:15:19.022410 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:19.022383 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678bw8jq_e94135bb-cabe-49f7-bf34-9220d510b9a9/istio-proxy/0.log" Jul 14 14:15:19.390449 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:19.390355 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-594bccb575-nmtvq_118356c5-d6ff-4662-b421-c2066cedec88/kube-auth-proxy/0.log" Jul 14 14:15:19.514838 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:19.514796 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-8wswz_c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92/istio-proxy/1.log" Jul 14 14:15:19.642373 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:19.642295 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-8nzwn_b517ab51-0cb0-47cb-8d09-476de327fe1a/payload-pre-processing/0.log" Jul 14 14:15:20.268630 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:20.268597 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-7djz8_ace2ce02-d8fd-4bf6-a7d2-94bb34db62c1/main/0.log" Jul 14 14:15:20.515419 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:20.515389 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-72x8w_037cf229-69eb-4a84-811b-b4eaf0d28fee/main/0.log" Jul 14 14:15:25.261370 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:25.261311 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:15:25.362285 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:25.362253 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:15:28.250487 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:28.250455 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-z9kpt_92645e2c-0c99-4eca-930a-686d53f52229/global-pull-secret-syncer/0.log" Jul 14 14:15:28.300890 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:28.300859 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" Jul 14 14:15:28.457242 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:28.457217 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-94.ec2.internal_e88c1491a47c596d63c06a067a091e22/haproxy/0.log" Jul 14 14:15:33.352176 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:33.352146 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-s7n6g_ec2f03ee-0cfb-47d2-92d6-04525e59aa26/manager/0.log" Jul 14 14:15:33.380121 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:33.380073 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-2nnxq_cbfcc3c6-4c72-40eb-a830-0b6b01c6947c/manager/0.log" Jul 14 14:15:33.457590 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:33.457560 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-67t8q_318072db-4ca4-4cd5-9b92-139d2abc7414/registry-server/0.log" Jul 14 14:15:33.571844 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:33.571749 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-pqbhm_6dea9fa8-c924-457d-bc4e-7477e5e8ff9f/manager/0.log" Jul 14 14:15:33.666416 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:33.666379 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-78tsv_2fa7beda-73cd-4e50-9a1d-0112b430a57c/manager/0.log" Jul 14 14:15:36.001969 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:36.001939 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rtnvv_1f971f01-724e-47f5-af52-16610accaf3a/node-exporter/0.log" Jul 14 14:15:36.024593 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:36.024566 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rtnvv_1f971f01-724e-47f5-af52-16610accaf3a/kube-rbac-proxy/0.log" Jul 14 14:15:36.047761 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:36.047735 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rtnvv_1f971f01-724e-47f5-af52-16610accaf3a/init-textfile/0.log" Jul 14 14:15:36.181821 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:36.181788 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_400f61c2-6d7a-4727-a3ca-350ddfed4218/prometheus/0.log" Jul 14 14:15:36.210790 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:36.210763 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_400f61c2-6d7a-4727-a3ca-350ddfed4218/config-reloader/0.log" Jul 14 14:15:36.237623 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:36.237597 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_400f61c2-6d7a-4727-a3ca-350ddfed4218/thanos-sidecar/0.log" Jul 14 14:15:36.262423 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:36.262344 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_400f61c2-6d7a-4727-a3ca-350ddfed4218/kube-rbac-proxy-web/0.log" Jul 14 14:15:36.285184 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:36.285161 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_400f61c2-6d7a-4727-a3ca-350ddfed4218/kube-rbac-proxy/0.log" Jul 14 14:15:36.310920 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:36.310894 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_400f61c2-6d7a-4727-a3ca-350ddfed4218/kube-rbac-proxy-thanos/0.log" Jul 14 14:15:36.341229 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:36.341203 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_400f61c2-6d7a-4727-a3ca-350ddfed4218/init-config-reloader/0.log" Jul 14 14:15:36.456041 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:36.456009 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78fbd8f94c-lpzzn_f52836e6-99fb-4f20-b546-df96573d560f/telemeter-client/0.log" Jul 14 14:15:36.480478 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:36.480455 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78fbd8f94c-lpzzn_f52836e6-99fb-4f20-b546-df96573d560f/reload/0.log" Jul 14 14:15:36.503890 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:36.503864 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78fbd8f94c-lpzzn_f52836e6-99fb-4f20-b546-df96573d560f/kube-rbac-proxy/0.log" Jul 14 14:15:38.123550 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.123513 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth"] Jul 14 14:15:38.123957 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.123942 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46003e01-fd2c-40c5-ae09-f1229ce68598" containerName="istio-proxy" Jul 14 14:15:38.123999 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.123959 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="46003e01-fd2c-40c5-ae09-f1229ce68598" containerName="istio-proxy" Jul 14 14:15:38.123999 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.123974 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39ea6257-1b7a-4bfe-92be-f9a728892902" containerName="istio-proxy" Jul 14 14:15:38.123999 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.123982 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="39ea6257-1b7a-4bfe-92be-f9a728892902" containerName="istio-proxy" Jul 14 14:15:38.123999 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.123990 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca" containerName="main" Jul 14 14:15:38.123999 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.123996 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca" containerName="main" Jul 14 14:15:38.124177 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.124009 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39ea6257-1b7a-4bfe-92be-f9a728892902" containerName="istio-proxy" Jul 14 14:15:38.124177 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.124015 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="39ea6257-1b7a-4bfe-92be-f9a728892902" containerName="istio-proxy" Jul 14 14:15:38.124177 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.124099 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="39ea6257-1b7a-4bfe-92be-f9a728892902" containerName="istio-proxy" Jul 14 14:15:38.124177 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.124107 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="46003e01-fd2c-40c5-ae09-f1229ce68598" containerName="istio-proxy" Jul 14 14:15:38.124177 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.124114 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="46003e01-fd2c-40c5-ae09-f1229ce68598" containerName="istio-proxy" Jul 14 14:15:38.124177 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.124120 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="39ea6257-1b7a-4bfe-92be-f9a728892902" containerName="istio-proxy" Jul 14 14:15:38.124177 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.124126 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e56e0ce0-8f2b-4556-8ee2-a1609e15b8ca" containerName="main" Jul 14 14:15:38.127302 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.127283 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" Jul 14 14:15:38.130113 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.130072 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t9pjq\"/\"kube-root-ca.crt\"" Jul 14 14:15:38.130252 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.130127 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t9pjq\"/\"openshift-service-ca.crt\"" Jul 14 14:15:38.131282 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.131260 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-t9pjq\"/\"default-dockercfg-gqsqg\"" Jul 14 14:15:38.137449 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.137427 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth"] Jul 14 14:15:38.182670 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.182642 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8b77c721-080d-4ae8-82a0-1be1975dd5da-proc\") pod \"perf-node-gather-daemonset-jqtth\" (UID: \"8b77c721-080d-4ae8-82a0-1be1975dd5da\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" Jul 14 14:15:38.182670 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.182673 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b77c721-080d-4ae8-82a0-1be1975dd5da-lib-modules\") pod \"perf-node-gather-daemonset-jqtth\" (UID: \"8b77c721-080d-4ae8-82a0-1be1975dd5da\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" Jul 14 14:15:38.182878 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.182715 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv2mm\" (UniqueName: \"kubernetes.io/projected/8b77c721-080d-4ae8-82a0-1be1975dd5da-kube-api-access-fv2mm\") pod \"perf-node-gather-daemonset-jqtth\" (UID: \"8b77c721-080d-4ae8-82a0-1be1975dd5da\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" Jul 14 14:15:38.182878 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.182750 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b77c721-080d-4ae8-82a0-1be1975dd5da-sys\") pod \"perf-node-gather-daemonset-jqtth\" (UID: \"8b77c721-080d-4ae8-82a0-1be1975dd5da\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" Jul 14 14:15:38.182878 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.182776 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8b77c721-080d-4ae8-82a0-1be1975dd5da-podres\") pod \"perf-node-gather-daemonset-jqtth\" (UID: \"8b77c721-080d-4ae8-82a0-1be1975dd5da\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" Jul 14 14:15:38.283904 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.283864 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8b77c721-080d-4ae8-82a0-1be1975dd5da-proc\") pod \"perf-node-gather-daemonset-jqtth\" (UID: \"8b77c721-080d-4ae8-82a0-1be1975dd5da\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" Jul 14 14:15:38.283904 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.283912 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b77c721-080d-4ae8-82a0-1be1975dd5da-lib-modules\") pod \"perf-node-gather-daemonset-jqtth\" (UID: \"8b77c721-080d-4ae8-82a0-1be1975dd5da\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" Jul 14 14:15:38.284207 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.283996 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv2mm\" (UniqueName: \"kubernetes.io/projected/8b77c721-080d-4ae8-82a0-1be1975dd5da-kube-api-access-fv2mm\") pod \"perf-node-gather-daemonset-jqtth\" (UID: \"8b77c721-080d-4ae8-82a0-1be1975dd5da\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" Jul 14 14:15:38.284207 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.284011 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8b77c721-080d-4ae8-82a0-1be1975dd5da-proc\") pod \"perf-node-gather-daemonset-jqtth\" (UID: \"8b77c721-080d-4ae8-82a0-1be1975dd5da\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" Jul 14 14:15:38.284207 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.284025 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b77c721-080d-4ae8-82a0-1be1975dd5da-sys\") pod \"perf-node-gather-daemonset-jqtth\" (UID: \"8b77c721-080d-4ae8-82a0-1be1975dd5da\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" Jul 14 14:15:38.284207 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.284103 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b77c721-080d-4ae8-82a0-1be1975dd5da-sys\") pod \"perf-node-gather-daemonset-jqtth\" (UID: \"8b77c721-080d-4ae8-82a0-1be1975dd5da\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" Jul 14 14:15:38.284207 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.284129 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8b77c721-080d-4ae8-82a0-1be1975dd5da-lib-modules\") pod \"perf-node-gather-daemonset-jqtth\" (UID: \"8b77c721-080d-4ae8-82a0-1be1975dd5da\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" Jul 14 14:15:38.284207 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.284132 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8b77c721-080d-4ae8-82a0-1be1975dd5da-podres\") pod \"perf-node-gather-daemonset-jqtth\" (UID: \"8b77c721-080d-4ae8-82a0-1be1975dd5da\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" Jul 14 14:15:38.284416 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.284212 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8b77c721-080d-4ae8-82a0-1be1975dd5da-podres\") pod \"perf-node-gather-daemonset-jqtth\" (UID: \"8b77c721-080d-4ae8-82a0-1be1975dd5da\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" Jul 14 14:15:38.293208 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.293181 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv2mm\" (UniqueName: \"kubernetes.io/projected/8b77c721-080d-4ae8-82a0-1be1975dd5da-kube-api-access-fv2mm\") pod \"perf-node-gather-daemonset-jqtth\" (UID: \"8b77c721-080d-4ae8-82a0-1be1975dd5da\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" Jul 14 14:15:38.438846 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.438764 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" Jul 14 14:15:38.578590 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.578564 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth"] Jul 14 14:15:38.581368 ip-10-0-142-94 kubenswrapper[2583]: W0714 14:15:38.581341 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8b77c721_080d_4ae8_82a0_1be1975dd5da.slice/crio-107974c99fb1aed7b1b386ade66f5bfd92746c43bf69b904fa16bc2ed166642a WatchSource:0}: Error finding container 107974c99fb1aed7b1b386ade66f5bfd92746c43bf69b904fa16bc2ed166642a: Status 404 returned error can't find the container with id 107974c99fb1aed7b1b386ade66f5bfd92746c43bf69b904fa16bc2ed166642a Jul 14 14:15:38.790990 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.790958 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" event={"ID":"8b77c721-080d-4ae8-82a0-1be1975dd5da","Type":"ContainerStarted","Data":"464846a9184f756d3e655c13aaf6b4e87f713b60d444ada29cc80fd55aed82bf"} Jul 14 14:15:38.791191 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.791009 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" event={"ID":"8b77c721-080d-4ae8-82a0-1be1975dd5da","Type":"ContainerStarted","Data":"107974c99fb1aed7b1b386ade66f5bfd92746c43bf69b904fa16bc2ed166642a"} Jul 14 14:15:38.791191 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.791095 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" Jul 14 14:15:38.809093 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:38.809040 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" podStartSLOduration=0.80902417 podStartE2EDuration="809.02417ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 14:15:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:15:38.807865289 +0000 UTC m=+2902.335015926" watchObservedRunningTime="2026-07-14 14:15:38.80902417 +0000 UTC m=+2902.336174863" Jul 14 14:15:40.746206 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:40.746173 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2zzrk_82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0/dns/0.log" Jul 14 14:15:40.776545 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:40.776522 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2zzrk_82d9fb6c-5cea-4c1f-8e02-8fe701a4d0a0/kube-rbac-proxy/0.log" Jul 14 14:15:40.947932 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:40.947899 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-plk87_02bc37b4-5dba-4801-92af-07beda478f46/dns-node-resolver/0.log" Jul 14 14:15:41.544499 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:41.544470 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dqvh8_6aec7895-aa6e-40c8-95ec-2b61bf996da0/node-ca/0.log" Jul 14 14:15:42.546742 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:42.546708 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678bw8jq_e94135bb-cabe-49f7-bf34-9220d510b9a9/istio-proxy/0.log" Jul 14 14:15:42.720048 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:42.720021 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-594bccb575-nmtvq_118356c5-d6ff-4662-b421-c2066cedec88/kube-auth-proxy/0.log" Jul 14 14:15:42.753464 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:42.753436 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-8wswz_c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92/istio-proxy/0.log" Jul 14 14:15:42.781140 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:42.781108 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-8wswz_c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92/istio-proxy/1.log" Jul 14 14:15:42.872125 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:42.872040 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-8nzwn_b517ab51-0cb0-47cb-8d09-476de327fe1a/payload-pre-processing/0.log" Jul 14 14:15:43.708777 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:43.708746 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2zllr_8125366b-15c5-4c38-a6ae-25101ed7473c/serve-healthcheck-canary/0.log" Jul 14 14:15:44.288790 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:44.288761 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jmjw8_6f321dd9-1f3d-42c0-b6eb-05130c73d8b9/kube-rbac-proxy/0.log" Jul 14 14:15:44.312428 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:44.312400 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jmjw8_6f321dd9-1f3d-42c0-b6eb-05130c73d8b9/exporter/0.log" Jul 14 14:15:44.338669 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:44.338639 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jmjw8_6f321dd9-1f3d-42c0-b6eb-05130c73d8b9/extractor/0.log" Jul 14 14:15:44.436015 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:44.435982 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:15:44.545723 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:44.545650 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-k24pk_4bc5e553-528f-46c0-8724-20c8c6d8024c/konnectivity-agent/0.log" limit=1024 Jul 14 14:15:44.805496 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:44.805425 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jqtth" Jul 14 14:15:46.675678 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:46.675645 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-qx7mp_109d56a5-7738-4b1f-ab5b-410e96a8aafb/manager/0.log" Jul 14 14:15:46.808108 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:46.808063 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-48fz7_18984890-7e5d-4556-aadc-1cc54a152630/manager/0.log" Jul 14 14:15:46.906409 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:46.906369 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-78bfdb59d-gtpx2_6aadcd44-2422-4e08-9bb0-cee729bd6011/manager/0.log" Jul 14 14:15:46.928405 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:46.928328 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gtnwl_0b8ca6b7-4548-463a-a764-cbd5f9a5dbdb/server/0.log" Jul 14 14:15:46.994131 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:46.994098 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-jx5df_218fb2da-1c3e-4b3b-984a-0861060202a3/manager/0.log" Jul 14 14:15:47.232556 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:47.232473 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9464f545f-xqxn6_20c64d8b-5726-4930-8be6-02866c1b10ca/manager/0.log" Jul 14 14:15:47.256626 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:47.256598 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9464f545f-xqxn6_20c64d8b-5726-4930-8be6-02866c1b10ca/copy-manifests/0.log" Jul 14 14:15:47.641667 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:47.641638 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678bw8jq_e94135bb-cabe-49f7-bf34-9220d510b9a9/istio-proxy/0.log" Jul 14 14:15:47.786379 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:47.786347 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-594bccb575-nmtvq_118356c5-d6ff-4662-b421-c2066cedec88/kube-auth-proxy/0.log" Jul 14 14:15:47.810505 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:47.810477 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-8wswz_c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92/istio-proxy/0.log" Jul 14 14:15:47.843211 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:47.843185 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-8wswz_c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92/istio-proxy/1.log" Jul 14 14:15:47.861747 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:47.861720 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-8nzwn_b517ab51-0cb0-47cb-8d09-476de327fe1a/payload-pre-processing/0.log" Jul 14 14:15:49.365902 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:49.365871 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678bw8jq_e94135bb-cabe-49f7-bf34-9220d510b9a9/istio-proxy/0.log" Jul 14 14:15:49.521288 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:49.521260 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-594bccb575-nmtvq_118356c5-d6ff-4662-b421-c2066cedec88/kube-auth-proxy/0.log" Jul 14 14:15:49.542601 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:49.542573 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-8wswz_c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92/istio-proxy/0.log" Jul 14 14:15:49.573276 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:49.573248 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-8wswz_c3bdbf8c-eadb-4aeb-ab2e-a5be43ad3a92/istio-proxy/1.log" Jul 14 14:15:49.593812 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:49.593788 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-8nzwn_b517ab51-0cb0-47cb-8d09-476de327fe1a/payload-pre-processing/0.log" Jul 14 14:15:53.802725 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:53.802693 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-b68dc_309f8182-1eb0-4ef2-8ff4-b33a08bde279/migrator/0.log" Jul 14 14:15:53.833747 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:53.833711 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-b68dc_309f8182-1eb0-4ef2-8ff4-b33a08bde279/graceful-termination/0.log" Jul 14 14:15:55.269944 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:55.269916 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7fmx5_86df2bc2-9305-4329-8bc4-53abd12c87ff/kube-multus/0.log" Jul 14 14:15:55.702032 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:55.701953 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xzjv6_56dc0a37-3391-46e8-b21d-71b167fd843f/kube-multus-additional-cni-plugins/0.log" Jul 14 14:15:55.727492 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:55.727465 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xzjv6_56dc0a37-3391-46e8-b21d-71b167fd843f/egress-router-binary-copy/0.log" Jul 14 14:15:55.752942 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:55.752923 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xzjv6_56dc0a37-3391-46e8-b21d-71b167fd843f/cni-plugins/0.log" Jul 14 14:15:55.775706 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:55.775684 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xzjv6_56dc0a37-3391-46e8-b21d-71b167fd843f/bond-cni-plugin/0.log" Jul 14 14:15:55.798679 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:55.798662 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xzjv6_56dc0a37-3391-46e8-b21d-71b167fd843f/routeoverride-cni/0.log" Jul 14 14:15:55.823788 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:55.823770 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xzjv6_56dc0a37-3391-46e8-b21d-71b167fd843f/whereabouts-cni-bincopy/0.log" Jul 14 14:15:55.847698 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:55.847669 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xzjv6_56dc0a37-3391-46e8-b21d-71b167fd843f/whereabouts-cni/0.log" Jul 14 14:15:56.077450 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:56.077412 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xrgrj_50a391d6-3c2b-478d-b005-4bbd65bdec74/network-metrics-daemon/0.log" Jul 14 14:15:56.098773 ip-10-0-142-94 kubenswrapper[2583]: I0714 14:15:56.098750 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xrgrj_50a391d6-3c2b-478d-b005-4bbd65bdec74/kube-rbac-proxy/0.log"