Jul 02 19:05:19.956019 ip-10-0-143-205 systemd[1]: Starting Kubernetes Kubelet... Jul 02 19:05:20.601687 ip-10-0-143-205 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 19:05:20.601687 ip-10-0-143-205 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 02 19:05:20.601687 ip-10-0-143-205 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 19:05:20.601687 ip-10-0-143-205 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 02 19:05:20.601687 ip-10-0-143-205 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 19:05:20.601687 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.586122 2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 02 19:05:20.601687 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.590975 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:05:20.601687 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.590986 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:05:20.601687 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.590990 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:05:20.601687 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.590993 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:05:20.601687 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.590996 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:05:20.601687 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.590999 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:05:20.601687 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591002 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:05:20.601687 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591005 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:05:20.603374 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591007 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:05:20.603374 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591010 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:05:20.603374 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591013 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:05:20.603374 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591016 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:05:20.603374 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591019 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:05:20.603374 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591022 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:05:20.603374 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591024 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:05:20.603374 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591027 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:05:20.603374 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591030 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:05:20.603374 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591041 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:05:20.603374 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591044 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:05:20.603374 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591047 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:05:20.603374 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591050 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:05:20.603374 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591053 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:05:20.603374 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591055 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:05:20.603374 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591058 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:05:20.603374 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591061 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:05:20.603374 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591063 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:05:20.603374 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591067 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:05:20.603374 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591070 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:05:20.604263 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591072 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:05:20.604263 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591075 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:05:20.604263 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591077 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:05:20.604263 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591080 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:05:20.604263 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591083 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:05:20.604263 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591086 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:05:20.604263 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591088 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:05:20.604263 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591090 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:05:20.604263 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591093 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:05:20.604263 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591096 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:05:20.604263 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591098 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:05:20.604263 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591101 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:05:20.604263 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591103 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:05:20.604263 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591106 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:05:20.604263 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591108 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:05:20.604263 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591111 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:05:20.604263 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591114 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:05:20.604263 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591117 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:05:20.604263 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591120 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:05:20.606019 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591122 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:05:20.606019 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591124 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:05:20.606019 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591127 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:05:20.606019 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591129 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:05:20.606019 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591132 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:05:20.606019 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591134 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:05:20.606019 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591137 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:05:20.606019 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591142 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:05:20.606019 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591146 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:05:20.606019 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591149 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:05:20.606019 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591152 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:05:20.606019 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591155 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:05:20.606019 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591158 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:05:20.606019 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591161 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:05:20.606019 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591164 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:05:20.606019 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591166 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:05:20.606019 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591169 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:05:20.606019 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591171 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:05:20.606019 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591174 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:05:20.606019 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591176 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:05:20.606897 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591179 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:05:20.606897 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591182 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:05:20.606897 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591184 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:05:20.606897 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591187 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:05:20.606897 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591189 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:05:20.606897 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591192 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:05:20.606897 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591196 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:05:20.606897 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591200 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:05:20.606897 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591204 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:05:20.606897 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591209 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:05:20.606897 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591213 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:05:20.606897 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591216 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:05:20.606897 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591219 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:05:20.606897 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591221 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:05:20.606897 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591224 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:05:20.606897 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591227 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:05:20.606897 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591229 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:05:20.606897 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591232 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:05:20.606897 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591235 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:05:20.608192 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591238 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:05:20.608192 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591240 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:05:20.608192 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591243 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:05:20.608192 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591246 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:05:20.608192 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591249 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:05:20.608192 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591251 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:05:20.608192 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591254 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:05:20.608192 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591257 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:05:20.608192 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591260 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:05:20.608192 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591262 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:05:20.608192 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591265 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:05:20.608192 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591267 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:05:20.608192 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591270 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:05:20.608192 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591971 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:05:20.608192 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591978 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:05:20.608192 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591981 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:05:20.608192 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591984 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:05:20.608192 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591987 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:05:20.608192 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591990 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:05:20.609083 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591992 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:05:20.609083 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591995 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:05:20.609083 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.591999 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:05:20.609083 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592002 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:05:20.609083 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592005 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:05:20.609083 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592008 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:05:20.609083 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592011 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:05:20.609083 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592014 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:05:20.609083 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592016 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:05:20.609083 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592019 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:05:20.609083 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592022 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:05:20.609083 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592024 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:05:20.609083 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592027 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:05:20.609083 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592030 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:05:20.609083 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592033 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:05:20.609083 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592036 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:05:20.609083 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592039 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:05:20.609083 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592042 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:05:20.609083 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592044 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:05:20.609083 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592047 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:05:20.610391 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592050 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:05:20.610391 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592052 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:05:20.610391 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592055 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:05:20.610391 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592057 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:05:20.610391 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592061 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:05:20.610391 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592063 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:05:20.610391 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592066 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:05:20.610391 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592069 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:05:20.610391 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592072 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:05:20.610391 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592074 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:05:20.610391 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592077 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:05:20.610391 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592080 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:05:20.610391 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592082 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:05:20.610391 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592085 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:05:20.610391 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592087 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:05:20.610391 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592090 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:05:20.610391 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592093 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:05:20.610391 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592095 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:05:20.610391 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592098 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:05:20.610391 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592100 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:05:20.612574 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592103 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:05:20.612574 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592106 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:05:20.612574 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592109 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:05:20.612574 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592111 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:05:20.612574 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592116 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:05:20.612574 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592121 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:05:20.612574 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592124 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:05:20.612574 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592127 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:05:20.612574 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592131 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:05:20.612574 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592134 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:05:20.612574 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592137 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:05:20.612574 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592140 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:05:20.612574 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592143 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:05:20.612574 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592145 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:05:20.612574 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592148 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:05:20.612574 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592151 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:05:20.612574 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592153 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:05:20.612574 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592156 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:05:20.612574 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592159 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:05:20.613123 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592161 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:05:20.613123 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592164 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:05:20.613123 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592166 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:05:20.613123 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592169 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:05:20.613123 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592172 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:05:20.613123 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592174 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:05:20.613123 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592177 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:05:20.613123 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592180 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:05:20.613123 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592182 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:05:20.613123 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592185 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:05:20.613123 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592189 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:05:20.613123 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592193 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:05:20.613123 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592196 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:05:20.613123 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592198 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:05:20.613123 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592201 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:05:20.613123 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592204 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:05:20.613123 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592206 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:05:20.613123 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592209 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:05:20.613123 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592212 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:05:20.613581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592215 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:05:20.613581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592217 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:05:20.613581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592220 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:05:20.613581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592222 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:05:20.613581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592226 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:05:20.613581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592230 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:05:20.613581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592233 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:05:20.613581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592236 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:05:20.613581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592239 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:05:20.613581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592242 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:05:20.613581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592246 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:05:20.613581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592248 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:05:20.613581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592251 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:05:20.613581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592254 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:05:20.613581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592256 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:05:20.613581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592639 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:05:20.613581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592643 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:05:20.613581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592646 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:05:20.613581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592649 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:05:20.614084 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592652 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:05:20.614084 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592655 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:05:20.614084 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592658 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:05:20.614084 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592661 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:05:20.614084 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592663 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:05:20.614084 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592666 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:05:20.614084 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592669 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:05:20.614084 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592672 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:05:20.614084 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592675 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:05:20.614084 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592677 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:05:20.614084 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592680 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:05:20.614084 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592682 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:05:20.614084 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592686 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:05:20.614084 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592688 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:05:20.614084 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592691 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:05:20.614084 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592693 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:05:20.614084 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592696 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:05:20.614084 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592699 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:05:20.614084 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592702 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:05:20.614084 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592705 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:05:20.614566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592723 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:05:20.614566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592727 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:05:20.614566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592733 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:05:20.614566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592735 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:05:20.614566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592738 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:05:20.614566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592741 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:05:20.614566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592743 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:05:20.614566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592746 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:05:20.614566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592749 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:05:20.614566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592752 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:05:20.614566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592754 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:05:20.614566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592757 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:05:20.614566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592760 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:05:20.614566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592762 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:05:20.614566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592765 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:05:20.614566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592767 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:05:20.614566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592771 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:05:20.614566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592774 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:05:20.614566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592776 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:05:20.614566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592779 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:05:20.615067 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592781 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:05:20.615067 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592784 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:05:20.615067 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592787 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:05:20.615067 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592792 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:05:20.615067 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592796 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:05:20.615067 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592800 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:05:20.615067 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592803 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:05:20.615067 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592806 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:05:20.615067 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592809 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:05:20.615067 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592812 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:05:20.615067 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592815 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:05:20.615067 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592818 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:05:20.615067 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592821 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:05:20.615067 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592824 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:05:20.615067 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592828 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:05:20.615067 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592832 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:05:20.615067 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592835 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:05:20.615067 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592838 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:05:20.615067 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592841 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:05:20.615573 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592844 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:05:20.615573 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592847 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:05:20.615573 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592850 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:05:20.615573 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592853 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:05:20.615573 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592856 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:05:20.615573 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592859 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:05:20.615573 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592862 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:05:20.615573 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592864 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:05:20.615573 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592867 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:05:20.615573 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592869 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:05:20.615573 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592872 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:05:20.615573 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592875 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:05:20.615573 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592878 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:05:20.615573 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592880 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:05:20.615573 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592883 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:05:20.615573 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592887 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:05:20.615573 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592891 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:05:20.615573 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592894 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:05:20.615573 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592897 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:05:20.615573 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592899 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:05:20.616078 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592902 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:05:20.616078 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592904 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:05:20.616078 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592907 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:05:20.616078 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592910 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:05:20.616078 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592912 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:05:20.616078 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592914 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:05:20.616078 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592917 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:05:20.616078 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592921 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:05:20.616078 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592924 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:05:20.616078 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592926 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:05:20.616078 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592929 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:05:20.616078 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592931 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:05:20.616078 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592934 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:05:20.616078 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592936 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:05:20.616078 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592939 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:05:20.616078 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.592941 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:05:20.616078 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593014 2566 flags.go:64] FLAG: --address="0.0.0.0" Jul 02 19:05:20.616078 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593021 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 02 19:05:20.616078 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593028 2566 flags.go:64] FLAG: --anonymous-auth="true" Jul 02 19:05:20.616078 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593035 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 02 19:05:20.616078 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593041 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593046 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593051 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593056 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593059 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593063 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593066 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593069 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593072 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593076 2566 flags.go:64] FLAG: --cgroup-root="" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593078 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593081 2566 flags.go:64] FLAG: --client-ca-file="" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593085 2566 flags.go:64] FLAG: --cloud-provider="external" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593088 2566 flags.go:64] FLAG: --cluster-dns="[]" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593092 2566 flags.go:64] FLAG: --cluster-domain="" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593095 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593098 2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593101 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593105 2566 flags.go:64] FLAG: --container-log-max-files="5" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593113 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593117 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593120 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593123 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593126 2566 flags.go:64] FLAG: --contention-profiling="false" Jul 02 19:05:20.616759 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593129 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593132 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593136 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593139 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593146 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593149 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593152 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593155 2566 flags.go:64] FLAG: --enable-load-reader="false" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593158 2566 flags.go:64] FLAG: --enable-server="true" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593161 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593165 2566 flags.go:64] FLAG: --event-burst="100" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593169 2566 flags.go:64] FLAG: --event-qps="50" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593172 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593175 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593178 2566 flags.go:64] FLAG: --eviction-hard="" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593182 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593185 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593188 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593191 2566 flags.go:64] FLAG: --eviction-soft="" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593193 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593196 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593199 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593202 2566 flags.go:64] FLAG: --experimental-mounter-path="" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593205 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593208 2566 flags.go:64] FLAG: --fail-swap-on="true" Jul 02 19:05:20.617418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593211 2566 flags.go:64] FLAG: --feature-gates="" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593215 2566 flags.go:64] FLAG: --file-check-frequency="20s" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593219 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593222 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593225 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593228 2566 flags.go:64] FLAG: --healthz-port="10248" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593231 2566 flags.go:64] FLAG: --help="false" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593235 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-143-205.ec2.internal" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593238 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593241 2566 flags.go:64] FLAG: --http-check-frequency="20s" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593244 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593248 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593252 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593255 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593258 2566 flags.go:64] FLAG: --image-service-endpoint="" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593261 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593264 2566 flags.go:64] FLAG: --kube-api-burst="100" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593267 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593270 2566 flags.go:64] FLAG: --kube-api-qps="50" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593273 2566 flags.go:64] FLAG: --kube-reserved="" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593277 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593280 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593283 2566 flags.go:64] FLAG: --kubelet-cgroups="" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593286 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 02 19:05:20.618071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593288 2566 flags.go:64] FLAG: --lock-file="" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593291 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593294 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593297 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593302 2566 flags.go:64] FLAG: --log-json-split-stream="false" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593305 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593308 2566 flags.go:64] FLAG: --log-text-split-stream="false" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593311 2566 flags.go:64] FLAG: --logging-format="text" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593313 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593317 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593321 2566 flags.go:64] FLAG: --manifest-url="" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593323 2566 flags.go:64] FLAG: --manifest-url-header="" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593328 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593331 2566 flags.go:64] FLAG: --max-open-files="1000000" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593334 2566 flags.go:64] FLAG: --max-pods="110" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593337 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593340 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593343 2566 flags.go:64] FLAG: --memory-manager-policy="None" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593346 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593351 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593354 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593356 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593363 2566 flags.go:64] FLAG: --node-status-max-images="50" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593366 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593369 2566 flags.go:64] FLAG: --oom-score-adj="-999" Jul 02 19:05:20.618728 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593372 2566 flags.go:64] FLAG: --pod-cidr="" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593375 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593381 2566 flags.go:64] FLAG: --pod-manifest-path="" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593383 2566 flags.go:64] FLAG: --pod-max-pids="-1" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593387 2566 flags.go:64] FLAG: --pods-per-core="0" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593390 2566 flags.go:64] FLAG: --port="10250" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593393 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593396 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f7b07887b85a24fc" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593399 2566 flags.go:64] FLAG: --qos-reserved="" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593402 2566 flags.go:64] FLAG: --read-only-port="10255" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593405 2566 flags.go:64] FLAG: --register-node="true" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593408 2566 flags.go:64] FLAG: --register-with-taints="" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593412 2566 flags.go:64] FLAG: --registry-burst="10" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593415 2566 flags.go:64] FLAG: --registry-qps="5" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593417 2566 flags.go:64] FLAG: --reserved-cpus="" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593420 2566 flags.go:64] FLAG: --reserved-memory="" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593424 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593428 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593431 2566 flags.go:64] FLAG: --rotate-certificates="false" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593434 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593437 2566 flags.go:64] FLAG: --runonce="false" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593439 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593442 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593445 2566 flags.go:64] FLAG: --seccomp-default="false" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593448 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 02 19:05:20.619465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593451 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593454 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593458 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593461 2566 flags.go:64] FLAG: --storage-driver-password="root" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593464 2566 flags.go:64] FLAG: --storage-driver-secure="false" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593467 2566 flags.go:64] FLAG: --storage-driver-table="stats" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593470 2566 flags.go:64] FLAG: --storage-driver-user="root" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593473 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593476 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593479 2566 flags.go:64] FLAG: --system-cgroups="" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593482 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593487 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593490 2566 flags.go:64] FLAG: --tls-cert-file="" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593494 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593498 2566 flags.go:64] FLAG: --tls-min-version="" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593501 2566 flags.go:64] FLAG: --tls-private-key-file="" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593503 2566 flags.go:64] FLAG: --topology-manager-policy="none" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593506 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593509 2566 flags.go:64] FLAG: --topology-manager-scope="container" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593512 2566 flags.go:64] FLAG: --v="2" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593516 2566 flags.go:64] FLAG: --version="false" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593520 2566 flags.go:64] FLAG: --vmodule="" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593523 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.593526 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593639 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:05:20.620188 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593643 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:05:20.620915 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593646 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:05:20.620915 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593649 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:05:20.620915 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593652 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:05:20.620915 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593655 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:05:20.620915 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593658 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:05:20.620915 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593661 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:05:20.620915 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593664 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:05:20.620915 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593666 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:05:20.620915 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593671 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:05:20.620915 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593673 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:05:20.620915 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593676 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:05:20.620915 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593679 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:05:20.620915 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593681 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:05:20.620915 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593684 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:05:20.620915 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593686 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:05:20.620915 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593689 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:05:20.620915 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593692 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:05:20.620915 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593696 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:05:20.620915 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593699 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:05:20.621461 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593703 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:05:20.621461 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593706 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:05:20.621461 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593726 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:05:20.621461 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593729 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:05:20.621461 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593731 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:05:20.621461 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593734 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:05:20.621461 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593737 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:05:20.621461 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593739 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:05:20.621461 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593742 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:05:20.621461 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593745 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:05:20.621461 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593747 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:05:20.621461 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593752 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:05:20.621461 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593754 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:05:20.621461 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593757 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:05:20.621461 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593759 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:05:20.621461 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593762 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:05:20.621461 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593764 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:05:20.621461 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593769 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:05:20.621461 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593772 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:05:20.622030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593776 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:05:20.622030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593779 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:05:20.622030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593783 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:05:20.622030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593786 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:05:20.622030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593789 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:05:20.622030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593791 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:05:20.622030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593794 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:05:20.622030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593797 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:05:20.622030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593799 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:05:20.622030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593802 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:05:20.622030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593805 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:05:20.622030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593807 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:05:20.622030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593810 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:05:20.622030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593813 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:05:20.622030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593816 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:05:20.622030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593819 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:05:20.622030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593822 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:05:20.622030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593824 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:05:20.622030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593827 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:05:20.622030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593830 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:05:20.622566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593832 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:05:20.622566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593835 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:05:20.622566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593838 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:05:20.622566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593840 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:05:20.622566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593844 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:05:20.622566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593846 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:05:20.622566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593849 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:05:20.622566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593851 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:05:20.622566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593854 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:05:20.622566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593857 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:05:20.622566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593859 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:05:20.622566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593862 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:05:20.622566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593865 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:05:20.622566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593868 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:05:20.622566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593873 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:05:20.622566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593876 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:05:20.622566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593879 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:05:20.622566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593882 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:05:20.622566 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593885 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:05:20.623115 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593888 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:05:20.623115 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593891 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:05:20.623115 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593894 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:05:20.623115 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593896 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:05:20.623115 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593899 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:05:20.623115 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593902 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:05:20.623115 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593905 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:05:20.623115 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593908 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:05:20.623115 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593911 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:05:20.623115 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593914 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:05:20.623115 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593917 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:05:20.623115 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593920 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:05:20.623115 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593922 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:05:20.623115 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593924 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:05:20.623115 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593927 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:05:20.623115 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593930 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:05:20.623115 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593932 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:05:20.623115 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593936 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:05:20.623115 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593938 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:05:20.623643 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.593941 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:05:20.623643 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.594511 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 19:05:20.623643 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.603545 2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 02 19:05:20.623643 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.603561 2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 02 19:05:20.623643 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.604386 2566 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 02 19:05:20.623643 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.604393 2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 02 19:05:20.623643 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604465 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:05:20.623643 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604472 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:05:20.623643 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604475 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:05:20.623643 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604479 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:05:20.623643 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604482 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:05:20.623643 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604486 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:05:20.623643 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604489 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:05:20.623643 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604492 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:05:20.624046 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604495 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:05:20.624046 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604498 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:05:20.624046 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604500 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:05:20.624046 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604503 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:05:20.624046 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604506 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:05:20.624046 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604508 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:05:20.624046 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604511 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:05:20.624046 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604514 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:05:20.624046 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604517 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:05:20.624046 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604520 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:05:20.624046 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604522 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:05:20.624046 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604525 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:05:20.624046 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604527 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:05:20.624046 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604531 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:05:20.624046 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604535 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:05:20.624046 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604539 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:05:20.624046 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604542 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:05:20.624046 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604546 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:05:20.624046 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604549 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:05:20.624046 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604552 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:05:20.624581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604555 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:05:20.624581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604558 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:05:20.624581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604561 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:05:20.624581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604563 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:05:20.624581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604566 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:05:20.624581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604569 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:05:20.624581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604571 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:05:20.624581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604574 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:05:20.624581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604577 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:05:20.624581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604580 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:05:20.624581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604582 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:05:20.624581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604585 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:05:20.624581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604588 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:05:20.624581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604591 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:05:20.624581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604593 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:05:20.624581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604596 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:05:20.624581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604598 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:05:20.624581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604601 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:05:20.624581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604603 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:05:20.624581 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604606 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:05:20.625092 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604609 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:05:20.625092 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604612 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:05:20.625092 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604614 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:05:20.625092 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604617 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:05:20.625092 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604619 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:05:20.625092 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604622 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:05:20.625092 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604625 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:05:20.625092 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604628 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:05:20.625092 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604630 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:05:20.625092 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604636 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:05:20.625092 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604640 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:05:20.625092 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604644 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:05:20.625092 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604647 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:05:20.625092 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604650 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:05:20.625092 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604653 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:05:20.625092 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604656 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:05:20.625092 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604659 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:05:20.625092 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604662 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:05:20.625092 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604664 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:05:20.625615 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604667 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:05:20.625615 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604670 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:05:20.625615 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604673 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:05:20.625615 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604676 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:05:20.625615 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604679 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:05:20.625615 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604682 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:05:20.625615 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604684 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:05:20.625615 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604687 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:05:20.625615 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604691 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:05:20.625615 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604695 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:05:20.625615 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604698 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:05:20.625615 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604701 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:05:20.625615 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604705 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:05:20.625615 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604718 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:05:20.625615 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604722 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:05:20.625615 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604724 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:05:20.625615 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604727 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:05:20.625615 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604729 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:05:20.625615 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604732 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:05:20.625615 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604735 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:05:20.626134 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604738 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:05:20.626134 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604740 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:05:20.626134 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604755 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:05:20.626134 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604758 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:05:20.626134 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604761 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:05:20.626134 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604764 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:05:20.626134 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604767 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:05:20.626134 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604769 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:05:20.626134 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604772 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:05:20.626134 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604775 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:05:20.626134 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604777 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:05:20.626134 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604780 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:05:20.626134 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.604785 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 19:05:20.626134 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604911 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 19:05:20.626134 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604916 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 19:05:20.626526 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604919 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 19:05:20.626526 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604921 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 19:05:20.626526 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604925 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 19:05:20.626526 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604927 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 19:05:20.626526 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604930 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 19:05:20.626526 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604933 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 19:05:20.626526 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604935 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 02 19:05:20.626526 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604939 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 19:05:20.626526 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604942 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 19:05:20.626526 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604944 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 19:05:20.626526 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604947 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 19:05:20.626526 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604950 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 19:05:20.626526 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604953 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 19:05:20.626526 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604955 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 19:05:20.626526 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604958 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 19:05:20.626526 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604961 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 19:05:20.626526 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604963 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 19:05:20.626526 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604966 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 19:05:20.626526 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604968 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 19:05:20.626526 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604972 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 19:05:20.627039 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604976 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 19:05:20.627039 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604979 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 19:05:20.627039 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604981 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 19:05:20.627039 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604984 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 19:05:20.627039 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604987 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 19:05:20.627039 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604989 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 19:05:20.627039 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604992 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 19:05:20.627039 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604995 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 19:05:20.627039 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.604997 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 19:05:20.627039 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605000 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 19:05:20.627039 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605002 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 19:05:20.627039 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605005 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 19:05:20.627039 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605008 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 19:05:20.627039 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605011 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 19:05:20.627039 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605014 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 19:05:20.627039 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605017 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 19:05:20.627039 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605019 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 19:05:20.627039 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605022 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 19:05:20.627039 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605025 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 19:05:20.627039 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605028 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 19:05:20.627632 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605031 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 19:05:20.627632 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605033 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 19:05:20.627632 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605036 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 19:05:20.627632 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605039 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 19:05:20.627632 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605042 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 19:05:20.627632 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605044 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 19:05:20.627632 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605047 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 19:05:20.627632 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605049 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 19:05:20.627632 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605052 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 19:05:20.627632 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605054 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 19:05:20.627632 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605057 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 19:05:20.627632 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605060 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 19:05:20.627632 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605064 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 19:05:20.627632 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605067 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 19:05:20.627632 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605070 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 19:05:20.627632 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605073 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 19:05:20.627632 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605076 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 19:05:20.627632 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605078 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 19:05:20.627632 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605081 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 19:05:20.628450 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605084 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 19:05:20.628450 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605086 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 19:05:20.628450 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605089 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 19:05:20.628450 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605092 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 19:05:20.628450 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605094 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 19:05:20.628450 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605097 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 19:05:20.628450 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605099 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 19:05:20.628450 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605102 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 19:05:20.628450 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605105 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 19:05:20.628450 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605107 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 19:05:20.628450 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605110 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 19:05:20.628450 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605112 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 19:05:20.628450 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605115 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 19:05:20.628450 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605118 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 19:05:20.628450 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605121 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 19:05:20.628450 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605123 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 19:05:20.628450 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605126 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 19:05:20.628450 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605130 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 19:05:20.628450 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605132 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 19:05:20.628450 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605137 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 19:05:20.629285 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605140 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 19:05:20.629285 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605143 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 19:05:20.629285 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605147 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 19:05:20.629285 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605151 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 19:05:20.629285 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605154 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 19:05:20.629285 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605158 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 19:05:20.629285 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605161 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 19:05:20.629285 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605163 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 19:05:20.629285 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605166 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 19:05:20.629285 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605169 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 19:05:20.629285 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605171 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 19:05:20.629285 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605174 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 19:05:20.629285 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605177 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 19:05:20.629285 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605179 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 19:05:20.629285 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605182 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 19:05:20.629285 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605184 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 19:05:20.629285 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605187 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 19:05:20.629285 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:20.605190 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 19:05:20.630083 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.605195 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 19:05:20.630083 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.605323 2566 server.go:962] "Client rotation is on, will bootstrap in background" Jul 02 19:05:20.630083 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.607807 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 02 19:05:20.630083 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.609381 2566 server.go:1019] "Starting client certificate rotation" Jul 02 19:05:20.630083 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.609482 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 19:05:20.630083 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.609519 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 19:05:20.630083 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.613442 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 19:05:20.630083 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.614745 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 19:05:20.630083 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.626975 2566 log.go:25] "Validated CRI v1 runtime API" Jul 02 19:05:20.630083 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.629583 2566 log.go:25] "Validated CRI v1 image API" Jul 02 19:05:20.630597 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.630567 2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 02 19:05:20.633936 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.633237 2566 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 bb65be28-35ee-4843-8491-9d9f3daaa850:/dev/nvme0n1p3 fad8abec-a879-4ae8-bdd7-9556f8ce3918:/dev/nvme0n1p4] Jul 02 19:05:20.634025 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.633953 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 02 19:05:20.639886 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.639729 2566 manager.go:216] Machine: {Timestamp:2026-07-02 19:05:20.638205517 +0000 UTC m=+0.565214226 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099452 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec224decf06ab05dc0e002d08bf55a0a SystemUUID:ec224dec-f06a-b05d-c0e0-02d08bf55a0a BootID:3d3437fe-6703-402b-a953-92eea5dfab2a Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:64:20:a9:d7:f9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:64:20:a9:d7:f9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:32:51:21:21:78:ac Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 02 19:05:20.639886 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.639876 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 02 19:05:20.640037 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.639995 2566 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 02 19:05:20.642408 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.642390 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 19:05:20.643511 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.643490 2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 02 19:05:20.643651 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.643515 2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-205.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 02 19:05:20.644269 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.644255 2566 topology_manager.go:138] "Creating topology manager with none policy" Jul 02 19:05:20.644269 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.644270 2566 container_manager_linux.go:309] "Creating device plugin manager" Jul 02 19:05:20.644774 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.644763 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 19:05:20.644807 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.644778 2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 19:05:20.646041 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.646031 2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 02 19:05:20.649371 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.649359 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 02 19:05:20.649483 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.649473 2566 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 02 19:05:20.649515 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.649504 2566 kubelet.go:486] "Attempting to sync node with API server" Jul 02 19:05:20.649515 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.649514 2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 02 19:05:20.649578 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.649525 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 02 19:05:20.649578 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.649534 2566 kubelet.go:398] "Adding apiserver pod source" Jul 02 19:05:20.649578 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.649543 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 02 19:05:20.649948 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.649921 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 19:05:20.650002 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.649957 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 19:05:20.652204 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.652181 2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 02 19:05:20.654208 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.654194 2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 02 19:05:20.654282 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.654224 2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 02 19:05:20.655319 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.655303 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 02 19:05:20.655319 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.655321 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 02 19:05:20.655431 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.655327 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 02 19:05:20.655431 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.655333 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 02 19:05:20.655431 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.655342 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 02 19:05:20.655431 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.655366 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 02 19:05:20.655431 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.655379 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 02 19:05:20.655431 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.655387 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 02 19:05:20.655431 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.655398 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 02 19:05:20.655431 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.655407 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 02 19:05:20.655431 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.655420 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 02 19:05:20.655431 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.655427 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 02 19:05:20.657227 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.657210 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 02 19:05:20.657311 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.657230 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 02 19:05:20.658972 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.658951 2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-205.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 02 19:05:20.659742 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:20.659721 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-205.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 02 19:05:20.659785 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:20.659730 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 02 19:05:20.660647 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.660632 2566 server.go:1268] "Started kubelet" Jul 02 19:05:20.660763 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.660738 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 02 19:05:20.661092 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.661043 2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 02 19:05:20.661187 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.661110 2566 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 02 19:05:20.662358 ip-10-0-143-205 systemd[1]: Started Kubernetes Kubelet. Jul 02 19:05:20.662600 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.662580 2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 02 19:05:20.663028 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.663005 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pzgdd" Jul 02 19:05:20.663934 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.663916 2566 server.go:310] "Adding debug handlers to kubelet server" Jul 02 19:05:20.668315 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:20.668289 2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 02 19:05:20.669129 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.669110 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pzgdd" Jul 02 19:05:20.671495 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.671480 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 02 19:05:20.671563 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.671497 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 02 19:05:20.671624 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:20.671584 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-205.ec2.internal\" not found" Jul 02 19:05:20.672002 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.671982 2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 02 19:05:20.672098 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.672015 2566 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 02 19:05:20.672098 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.672034 2566 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 02 19:05:20.672217 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.672177 2566 reconstruct.go:97] "Volume reconstruction finished" Jul 02 19:05:20.672217 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.672185 2566 reconciler.go:29] "Reconciler: start to sync state" Jul 02 19:05:20.672572 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.672553 2566 factory.go:55] Registering systemd factory Jul 02 19:05:20.672572 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.672575 2566 factory.go:223] Registration of the systemd container factory successfully Jul 02 19:05:20.673403 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.673373 2566 factory.go:153] Registering CRI-O factory Jul 02 19:05:20.673403 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.673400 2566 factory.go:223] Registration of the crio container factory successfully Jul 02 19:05:20.673549 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.673453 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 02 19:05:20.673549 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.673477 2566 factory.go:103] Registering Raw factory Jul 02 19:05:20.673549 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.673493 2566 manager.go:1196] Started watching for new ooms in manager Jul 02 19:05:20.674406 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.674390 2566 manager.go:349] Starting recovery of all containers Jul 02 19:05:20.675888 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.675866 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:05:20.680564 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:20.680539 2566 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-205.ec2.internal\" not found" node="ip-10-0-143-205.ec2.internal" Jul 02 19:05:20.687100 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.687083 2566 manager.go:354] Recovery completed Jul 02 19:05:20.691862 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.691849 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:05:20.694194 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.694175 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:05:20.694266 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.694206 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:05:20.694266 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.694220 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:05:20.694715 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.694704 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:05:20.694773 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.694718 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 02 19:05:20.694773 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.694727 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 02 19:05:20.694773 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.694743 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 02 19:05:20.695433 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.695420 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:05:20.695497 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.695446 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:05:20.695497 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.695459 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:05:20.695848 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.695837 2566 policy_none.go:49] "None policy: Start" Jul 02 19:05:20.695883 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.695853 2566 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 02 19:05:20.695883 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.695863 2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 02 19:05:20.696806 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.696796 2566 policy_none.go:47] "Start" Jul 02 19:05:20.732456 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.732284 2566 manager.go:337] "Starting Device Plugin manager" Jul 02 19:05:20.732556 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:20.732497 2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 02 19:05:20.732556 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.732514 2566 server.go:87] "Starting device plugin registration server" Jul 02 19:05:20.732811 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.732799 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 02 19:05:20.732847 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.732814 2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 02 19:05:20.732955 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.732912 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 02 19:05:20.733047 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.732998 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 02 19:05:20.733047 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.733008 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 02 19:05:20.733595 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:20.733574 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 02 19:05:20.733660 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:20.733632 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-205.ec2.internal\" not found" Jul 02 19:05:20.833783 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.833753 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:05:20.834920 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.834778 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:05:20.834920 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.834818 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:05:20.834920 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.834836 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:05:20.834920 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.834872 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-205.ec2.internal" Jul 02 19:05:20.836209 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.836182 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 02 19:05:20.837463 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.837445 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 02 19:05:20.837555 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.837491 2566 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 02 19:05:20.837555 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.837516 2566 kubelet.go:2448] "Starting kubelet main sync loop" Jul 02 19:05:20.837650 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:20.837555 2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 02 19:05:20.839744 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.839725 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:05:20.843951 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.843934 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-205.ec2.internal" Jul 02 19:05:20.844044 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:20.843959 2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-205.ec2.internal\": node \"ip-10-0-143-205.ec2.internal\" not found" Jul 02 19:05:20.893295 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:20.893232 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-205.ec2.internal\" not found" Jul 02 19:05:20.938474 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.938441 2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-205.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-205.ec2.internal"] Jul 02 19:05:20.938557 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.938519 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:05:20.940098 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.940081 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:05:20.940192 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.940116 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:05:20.940192 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.940130 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:05:20.941559 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.941543 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:05:20.941674 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.941658 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-205.ec2.internal" Jul 02 19:05:20.941742 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.941705 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:05:20.942295 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.942273 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:05:20.942295 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.942286 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:05:20.942405 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.942301 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:05:20.942405 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.942308 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:05:20.942405 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.942311 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:05:20.942405 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.942319 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:05:20.943948 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.943934 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-205.ec2.internal" Jul 02 19:05:20.944007 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.943960 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 19:05:20.944638 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.944623 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeHasSufficientMemory" Jul 02 19:05:20.944732 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.944649 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 19:05:20.944732 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.944679 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeHasSufficientPID" Jul 02 19:05:20.968317 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:20.968294 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-205.ec2.internal\" not found" node="ip-10-0-143-205.ec2.internal" Jul 02 19:05:20.972721 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:20.972698 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-205.ec2.internal\" not found" node="ip-10-0-143-205.ec2.internal" Jul 02 19:05:20.974806 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.974791 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/443b95f9a11c785c66a6608d9d6fba98-config\") pod \"kube-apiserver-proxy-ip-10-0-143-205.ec2.internal\" (UID: \"443b95f9a11c785c66a6608d9d6fba98\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-205.ec2.internal" Jul 02 19:05:20.974856 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.974814 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/98feeb3b547af7c850dfc4e76f035021-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-205.ec2.internal\" (UID: \"98feeb3b547af7c850dfc4e76f035021\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-205.ec2.internal" Jul 02 19:05:20.974856 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:20.974832 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98feeb3b547af7c850dfc4e76f035021-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-205.ec2.internal\" (UID: \"98feeb3b547af7c850dfc4e76f035021\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-205.ec2.internal" Jul 02 19:05:20.993866 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:20.993849 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-205.ec2.internal\" not found" Jul 02 19:05:21.075388 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:21.075357 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/98feeb3b547af7c850dfc4e76f035021-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-205.ec2.internal\" (UID: \"98feeb3b547af7c850dfc4e76f035021\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-205.ec2.internal" Jul 02 19:05:21.075870 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:21.075387 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98feeb3b547af7c850dfc4e76f035021-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-205.ec2.internal\" (UID: \"98feeb3b547af7c850dfc4e76f035021\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-205.ec2.internal" Jul 02 19:05:21.075870 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:21.075774 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98feeb3b547af7c850dfc4e76f035021-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-205.ec2.internal\" (UID: \"98feeb3b547af7c850dfc4e76f035021\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-205.ec2.internal" Jul 02 19:05:21.075870 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:21.075864 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/443b95f9a11c785c66a6608d9d6fba98-config\") pod \"kube-apiserver-proxy-ip-10-0-143-205.ec2.internal\" (UID: \"443b95f9a11c785c66a6608d9d6fba98\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-205.ec2.internal" Jul 02 19:05:21.077405 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:21.075465 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/98feeb3b547af7c850dfc4e76f035021-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-205.ec2.internal\" (UID: \"98feeb3b547af7c850dfc4e76f035021\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-205.ec2.internal" Jul 02 19:05:21.077405 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:21.075925 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/443b95f9a11c785c66a6608d9d6fba98-config\") pod \"kube-apiserver-proxy-ip-10-0-143-205.ec2.internal\" (UID: \"443b95f9a11c785c66a6608d9d6fba98\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-205.ec2.internal" Jul 02 19:05:21.094034 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:21.094007 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-205.ec2.internal\" not found" Jul 02 19:05:21.194235 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:21.194140 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-205.ec2.internal\" not found" Jul 02 19:05:21.273229 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:21.273181 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-205.ec2.internal" Jul 02 19:05:21.275383 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:21.275368 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-205.ec2.internal" Jul 02 19:05:21.294940 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:21.294919 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-205.ec2.internal\" not found" Jul 02 19:05:21.395562 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:21.395521 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-205.ec2.internal\" not found" Jul 02 19:05:21.496517 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:21.496449 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-205.ec2.internal\" not found" Jul 02 19:05:21.597041 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:21.597007 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-205.ec2.internal\" not found" Jul 02 19:05:21.608304 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:21.608276 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 02 19:05:21.608778 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:21.608461 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 19:05:21.608778 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:21.608468 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 19:05:21.673404 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:21.673337 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 19:00:20 +0000 UTC" deadline="2027-11-30 15:59:28.403071666 +0000 UTC" Jul 02 19:05:21.673404 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:21.673396 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12380h54m6.729679014s" Jul 02 19:05:21.697717 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:21.697689 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-205.ec2.internal\" not found" Jul 02 19:05:21.798376 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:21.798346 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-205.ec2.internal\" not found" Jul 02 19:05:21.815595 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:21.815564 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:05:21.869980 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:21.869943 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98feeb3b547af7c850dfc4e76f035021.slice/crio-12d2a40a896940e67ad0dd6a4c8bc033833ce29daaaa9381a7a76e51c33eaed4 WatchSource:0}: Error finding container 12d2a40a896940e67ad0dd6a4c8bc033833ce29daaaa9381a7a76e51c33eaed4: Status 404 returned error can't find the container with id 12d2a40a896940e67ad0dd6a4c8bc033833ce29daaaa9381a7a76e51c33eaed4 Jul 02 19:05:21.870470 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:21.870449 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod443b95f9a11c785c66a6608d9d6fba98.slice/crio-733017917baaba2552b2f6231c8cd4568b0aea6d1a6b012044563624158fc7ae WatchSource:0}: Error finding container 733017917baaba2552b2f6231c8cd4568b0aea6d1a6b012044563624158fc7ae: Status 404 returned error can't find the container with id 733017917baaba2552b2f6231c8cd4568b0aea6d1a6b012044563624158fc7ae Jul 02 19:05:21.875509 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:21.875494 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 19:05:21.899487 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:21.899466 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-205.ec2.internal\" not found" Jul 02 19:05:21.999932 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:21.999889 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-205.ec2.internal\" not found" Jul 02 19:05:22.100453 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:22.100355 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-205.ec2.internal\" not found" Jul 02 19:05:22.188018 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.187989 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:05:22.272144 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.272115 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-205.ec2.internal" Jul 02 19:05:22.281429 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.281406 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 19:05:22.282425 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.282412 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-205.ec2.internal" Jul 02 19:05:22.292049 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.292031 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 19:05:22.628795 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.628765 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:05:22.651038 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.651011 2566 apiserver.go:52] "Watching apiserver" Jul 02 19:05:22.657776 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.657753 2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 02 19:05:22.659041 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.659009 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-205.ec2.internal","openshift-multus/multus-additional-cni-plugins-6nnrq","openshift-multus/multus-z8v4v","openshift-network-operator/iptables-alerter-kpjfw","kube-system/konnectivity-agent-bl8kt","kube-system/kube-apiserver-proxy-ip-10-0-143-205.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg","openshift-multus/network-metrics-daemon-c9b6h","openshift-network-diagnostics/network-check-target-hssx4","openshift-ovn-kubernetes/ovnkube-node-29hfm","kube-system/global-pull-secret-syncer-txtgt","openshift-cluster-node-tuning-operator/tuned-7hnbs","openshift-dns/node-resolver-hbf5k","openshift-image-registry/node-ca-45z57"] Jul 02 19:05:22.660624 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.660598 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.661822 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.661801 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hbf5k" Jul 02 19:05:22.663375 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.662936 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 02 19:05:22.663375 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.662972 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-45z57" Jul 02 19:05:22.663375 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.663264 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 02 19:05:22.663375 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.663348 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pv6g2\"" Jul 02 19:05:22.663589 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.663396 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 02 19:05:22.665594 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.663946 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 02 19:05:22.665594 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.664037 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 02 19:05:22.665594 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.664046 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g5jlc\"" Jul 02 19:05:22.665594 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.664235 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kpjfw" Jul 02 19:05:22.665594 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.664832 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 02 19:05:22.665594 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.665097 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 02 19:05:22.665594 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.665197 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rzc27\"" Jul 02 19:05:22.665594 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.665209 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 02 19:05:22.666021 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.665930 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bl8kt" Jul 02 19:05:22.666574 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.666551 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 02 19:05:22.666662 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.666554 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 02 19:05:22.670392 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.670368 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 02 19:05:22.670480 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.670450 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-w65xh\"" Jul 02 19:05:22.670669 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.670653 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:22.670749 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.670702 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 02 19:05:22.670801 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:22.670761 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9b6h" podUID="92620d26-b178-4330-a011-98af64b4d43c" Jul 02 19:05:22.673163 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.670862 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 02 19:05:22.673163 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.670972 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:22.673163 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:22.671046 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hssx4" podUID="4d87225d-39f9-40d0-8ae7-5fa11099c16c" Jul 02 19:05:22.673163 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.671233 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5twqp\"" Jul 02 19:05:22.673163 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.671538 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.673163 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.672232 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 02 19:05:22.674312 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.673707 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.674889 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.674534 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 02 19:05:22.674889 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.674608 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 02 19:05:22.674889 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.674536 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-v25b2\"" Jul 02 19:05:22.675977 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.675960 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.676545 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.676520 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 02 19:05:22.676981 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.676965 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 02 19:05:22.677241 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.677220 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 02 19:05:22.677327 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.677295 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 02 19:05:22.677475 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.677458 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zrvzn\"" Jul 02 19:05:22.677541 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.677495 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 02 19:05:22.678266 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.678249 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:22.678394 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:22.678368 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txtgt" podUID="0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef" Jul 02 19:05:22.678569 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.678533 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 02 19:05:22.679001 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.678982 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 02 19:05:22.679088 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.679017 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 02 19:05:22.679165 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.679142 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5v5dv\"" Jul 02 19:05:22.679340 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.679323 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 02 19:05:22.679474 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.679458 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 02 19:05:22.679559 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.679540 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 02 19:05:22.681243 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.681224 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.683929 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.683805 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 02 19:05:22.684486 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.684467 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-t9xkw\"" Jul 02 19:05:22.685041 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685006 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-system-cni-dir\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.685148 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685040 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-cnibin\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.685148 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685063 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e-host\") pod \"node-ca-45z57\" (UID: \"d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e\") " pod="openshift-image-registry/node-ca-45z57" Jul 02 19:05:22.685148 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685087 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69f9s\" (UniqueName: \"kubernetes.io/projected/d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e-kube-api-access-69f9s\") pod \"node-ca-45z57\" (UID: \"d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e\") " pod="openshift-image-registry/node-ca-45z57" Jul 02 19:05:22.685148 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685128 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a61e4596-f5a1-439d-a1e5-909708c4a2a0-konnectivity-ca\") pod \"konnectivity-agent-bl8kt\" (UID: \"a61e4596-f5a1-439d-a1e5-909708c4a2a0\") " pod="kube-system/konnectivity-agent-bl8kt" Jul 02 19:05:22.685381 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685152 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhqhb\" (UniqueName: \"kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb\") pod \"network-check-target-hssx4\" (UID: \"4d87225d-39f9-40d0-8ae7-5fa11099c16c\") " pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:22.685381 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685174 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3cf388a6-8721-429f-a2f9-f5f950e40a3b-registration-dir\") pod \"aws-ebs-csi-driver-node-9mbwg\" (UID: \"3cf388a6-8721-429f-a2f9-f5f950e40a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.685381 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685199 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/719b7248-85a7-4687-a8be-e806e338165f-etc-tuned\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.685381 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685270 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9p58\" (UniqueName: \"kubernetes.io/projected/719b7248-85a7-4687-a8be-e806e338165f-kube-api-access-n9p58\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.685381 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685304 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhzv5\" (UniqueName: \"kubernetes.io/projected/bcd6dcd7-7601-48bc-a635-b32b28d95a64-kube-api-access-jhzv5\") pod \"iptables-alerter-kpjfw\" (UID: \"bcd6dcd7-7601-48bc-a635-b32b28d95a64\") " pod="openshift-network-operator/iptables-alerter-kpjfw" Jul 02 19:05:22.685381 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685331 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbfqb\" (UniqueName: \"kubernetes.io/projected/92620d26-b178-4330-a011-98af64b4d43c-kube-api-access-pbfqb\") pod \"network-metrics-daemon-c9b6h\" (UID: \"92620d26-b178-4330-a011-98af64b4d43c\") " pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:22.685381 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685355 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.685738 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685472 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.685738 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685521 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n9p2\" (UniqueName: \"kubernetes.io/projected/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-kube-api-access-9n9p2\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.685738 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685551 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-sys\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.685738 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685576 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a04cbb97-93a7-4d61-bdf9-deba1968e7c0-tmp-dir\") pod \"node-resolver-hbf5k\" (UID: \"a04cbb97-93a7-4d61-bdf9-deba1968e7c0\") " pod="openshift-dns/node-resolver-hbf5k" Jul 02 19:05:22.685738 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685599 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfx2z\" (UniqueName: \"kubernetes.io/projected/a04cbb97-93a7-4d61-bdf9-deba1968e7c0-kube-api-access-wfx2z\") pod \"node-resolver-hbf5k\" (UID: \"a04cbb97-93a7-4d61-bdf9-deba1968e7c0\") " pod="openshift-dns/node-resolver-hbf5k" Jul 02 19:05:22.685738 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685622 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-cni-binary-copy\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.685738 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685648 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3cf388a6-8721-429f-a2f9-f5f950e40a3b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9mbwg\" (UID: \"3cf388a6-8721-429f-a2f9-f5f950e40a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.685738 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685718 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3cf388a6-8721-429f-a2f9-f5f950e40a3b-sys-fs\") pod \"aws-ebs-csi-driver-node-9mbwg\" (UID: \"3cf388a6-8721-429f-a2f9-f5f950e40a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.686194 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685750 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6ftx\" (UniqueName: \"kubernetes.io/projected/3cf388a6-8721-429f-a2f9-f5f950e40a3b-kube-api-access-l6ftx\") pod \"aws-ebs-csi-driver-node-9mbwg\" (UID: \"3cf388a6-8721-429f-a2f9-f5f950e40a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.686194 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685779 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e-serviceca\") pod \"node-ca-45z57\" (UID: \"d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e\") " pod="openshift-image-registry/node-ca-45z57" Jul 02 19:05:22.686194 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685803 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-etc-modprobe-d\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.686194 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685859 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-lib-modules\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.686194 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685886 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bcd6dcd7-7601-48bc-a635-b32b28d95a64-iptables-alerter-script\") pod \"iptables-alerter-kpjfw\" (UID: \"bcd6dcd7-7601-48bc-a635-b32b28d95a64\") " pod="openshift-network-operator/iptables-alerter-kpjfw" Jul 02 19:05:22.686194 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685911 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs\") pod \"network-metrics-daemon-c9b6h\" (UID: \"92620d26-b178-4330-a011-98af64b4d43c\") " pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:22.686194 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685937 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-os-release\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.686194 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685961 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a61e4596-f5a1-439d-a1e5-909708c4a2a0-agent-certs\") pod \"konnectivity-agent-bl8kt\" (UID: \"a61e4596-f5a1-439d-a1e5-909708c4a2a0\") " pod="kube-system/konnectivity-agent-bl8kt" Jul 02 19:05:22.686194 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.685987 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-var-lib-kubelet\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.686194 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.686052 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3cf388a6-8721-429f-a2f9-f5f950e40a3b-etc-selinux\") pod \"aws-ebs-csi-driver-node-9mbwg\" (UID: \"3cf388a6-8721-429f-a2f9-f5f950e40a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.686194 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.686099 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-etc-sysctl-d\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.686194 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.686144 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-run\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.686194 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.686176 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/719b7248-85a7-4687-a8be-e806e338165f-tmp\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.686807 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.686203 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-etc-sysconfig\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.686807 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.686241 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-etc-sysctl-conf\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.686807 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.686269 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-host\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.686807 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.686372 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a04cbb97-93a7-4d61-bdf9-deba1968e7c0-hosts-file\") pod \"node-resolver-hbf5k\" (UID: \"a04cbb97-93a7-4d61-bdf9-deba1968e7c0\") " pod="openshift-dns/node-resolver-hbf5k" Jul 02 19:05:22.686807 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.686403 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bcd6dcd7-7601-48bc-a635-b32b28d95a64-host-slash\") pod \"iptables-alerter-kpjfw\" (UID: \"bcd6dcd7-7601-48bc-a635-b32b28d95a64\") " pod="openshift-network-operator/iptables-alerter-kpjfw" Jul 02 19:05:22.686807 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.686428 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.686807 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.686459 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3cf388a6-8721-429f-a2f9-f5f950e40a3b-socket-dir\") pod \"aws-ebs-csi-driver-node-9mbwg\" (UID: \"3cf388a6-8721-429f-a2f9-f5f950e40a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.686807 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.686481 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3cf388a6-8721-429f-a2f9-f5f950e40a3b-device-dir\") pod \"aws-ebs-csi-driver-node-9mbwg\" (UID: \"3cf388a6-8721-429f-a2f9-f5f950e40a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.686807 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.686503 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-etc-kubernetes\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.686807 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.686526 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-etc-systemd\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.687406 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.687384 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 19:05:22.719415 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.719334 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-j7pck" Jul 02 19:05:22.734801 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.734777 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-j7pck" Jul 02 19:05:22.773705 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.773667 2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 02 19:05:22.787588 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.787558 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-systemd-units\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.787588 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.787589 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-host-cni-netd\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.787775 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.787630 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-etc-systemd\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.787775 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.787662 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-etc-systemd\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.787775 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.787696 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-system-cni-dir\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.787775 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.787714 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-cnibin\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.787775 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.787738 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-host-cni-bin\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.787775 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.787755 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/85a38081-5d7f-444a-837e-f536c6049157-ovnkube-script-lib\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.788008 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.787789 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/534a1f91-37b3-4ba4-93fc-f872ed2b668f-cni-binary-copy\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.788008 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.787794 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-system-cni-dir\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.788008 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.787809 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-etc-kubernetes\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.788008 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.787817 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-cnibin\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.788008 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.787861 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhqhb\" (UniqueName: \"kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb\") pod \"network-check-target-hssx4\" (UID: \"4d87225d-39f9-40d0-8ae7-5fa11099c16c\") " pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:22.788008 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.787881 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3cf388a6-8721-429f-a2f9-f5f950e40a3b-registration-dir\") pod \"aws-ebs-csi-driver-node-9mbwg\" (UID: \"3cf388a6-8721-429f-a2f9-f5f950e40a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.788008 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.787897 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-var-lib-openvswitch\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.788008 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.787953 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3cf388a6-8721-429f-a2f9-f5f950e40a3b-registration-dir\") pod \"aws-ebs-csi-driver-node-9mbwg\" (UID: \"3cf388a6-8721-429f-a2f9-f5f950e40a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.788008 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.787954 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.788450 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788042 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/719b7248-85a7-4687-a8be-e806e338165f-etc-tuned\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.788450 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788087 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9p58\" (UniqueName: \"kubernetes.io/projected/719b7248-85a7-4687-a8be-e806e338165f-kube-api-access-n9p58\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.788450 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788112 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhzv5\" (UniqueName: \"kubernetes.io/projected/bcd6dcd7-7601-48bc-a635-b32b28d95a64-kube-api-access-jhzv5\") pod \"iptables-alerter-kpjfw\" (UID: \"bcd6dcd7-7601-48bc-a635-b32b28d95a64\") " pod="openshift-network-operator/iptables-alerter-kpjfw" Jul 02 19:05:22.788450 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788135 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n9p2\" (UniqueName: \"kubernetes.io/projected/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-kube-api-access-9n9p2\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.788450 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788176 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e-serviceca\") pod \"node-ca-45z57\" (UID: \"d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e\") " pod="openshift-image-registry/node-ca-45z57" Jul 02 19:05:22.788450 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788194 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-host-kubelet\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.788450 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788217 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-host-run-ovn-kubernetes\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.788450 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788244 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-sys\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.788450 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788269 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a04cbb97-93a7-4d61-bdf9-deba1968e7c0-tmp-dir\") pod \"node-resolver-hbf5k\" (UID: \"a04cbb97-93a7-4d61-bdf9-deba1968e7c0\") " pod="openshift-dns/node-resolver-hbf5k" Jul 02 19:05:22.788450 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788295 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfx2z\" (UniqueName: \"kubernetes.io/projected/a04cbb97-93a7-4d61-bdf9-deba1968e7c0-kube-api-access-wfx2z\") pod \"node-resolver-hbf5k\" (UID: \"a04cbb97-93a7-4d61-bdf9-deba1968e7c0\") " pod="openshift-dns/node-resolver-hbf5k" Jul 02 19:05:22.788450 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788325 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a61e4596-f5a1-439d-a1e5-909708c4a2a0-agent-certs\") pod \"konnectivity-agent-bl8kt\" (UID: \"a61e4596-f5a1-439d-a1e5-909708c4a2a0\") " pod="kube-system/konnectivity-agent-bl8kt" Jul 02 19:05:22.788450 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788349 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-run-systemd\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.788450 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788373 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-run-openvswitch\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.788450 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788397 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/85a38081-5d7f-444a-837e-f536c6049157-ovn-node-metrics-cert\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.788450 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788426 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-cnibin\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.789122 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788470 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-etc-modprobe-d\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.789122 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788497 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs\") pod \"network-metrics-daemon-c9b6h\" (UID: \"92620d26-b178-4330-a011-98af64b4d43c\") " pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:22.789122 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788480 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 02 19:05:22.789122 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788540 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-os-release\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.789122 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788635 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-multus-socket-dir-parent\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.789122 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788646 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-sys\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.789122 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788647 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-os-release\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.789122 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788665 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-host-run-multus-certs\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.789122 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788735 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a04cbb97-93a7-4d61-bdf9-deba1968e7c0-tmp-dir\") pod \"node-resolver-hbf5k\" (UID: \"a04cbb97-93a7-4d61-bdf9-deba1968e7c0\") " pod="openshift-dns/node-resolver-hbf5k" Jul 02 19:05:22.789122 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788752 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e-serviceca\") pod \"node-ca-45z57\" (UID: \"d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e\") " pod="openshift-image-registry/node-ca-45z57" Jul 02 19:05:22.789122 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788793 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-host-run-k8s-cni-cncf-io\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.789122 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:22.788899 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:05:22.789122 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788926 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-etc-modprobe-d\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.789122 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.788938 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-etc-sysctl-d\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.789122 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:22.789015 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs podName:92620d26-b178-4330-a011-98af64b4d43c nodeName:}" failed. No retries permitted until 2026-07-02 19:05:23.288973916 +0000 UTC m=+3.215982622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs") pod "network-metrics-daemon-c9b6h" (UID: "92620d26-b178-4330-a011-98af64b4d43c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:05:22.789122 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.789039 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-run-ovn\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.789122 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.789072 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-node-log\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.789780 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.789121 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-etc-sysctl-d\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.789780 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.789131 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-host\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.789780 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.789192 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a04cbb97-93a7-4d61-bdf9-deba1968e7c0-hosts-file\") pod \"node-resolver-hbf5k\" (UID: \"a04cbb97-93a7-4d61-bdf9-deba1968e7c0\") " pod="openshift-dns/node-resolver-hbf5k" Jul 02 19:05:22.789780 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.789205 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-host\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.789780 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.789232 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bcd6dcd7-7601-48bc-a635-b32b28d95a64-host-slash\") pod \"iptables-alerter-kpjfw\" (UID: \"bcd6dcd7-7601-48bc-a635-b32b28d95a64\") " pod="openshift-network-operator/iptables-alerter-kpjfw" Jul 02 19:05:22.789780 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.789269 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a04cbb97-93a7-4d61-bdf9-deba1968e7c0-hosts-file\") pod \"node-resolver-hbf5k\" (UID: \"a04cbb97-93a7-4d61-bdf9-deba1968e7c0\") " pod="openshift-dns/node-resolver-hbf5k" Jul 02 19:05:22.789780 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.789307 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a61e4596-f5a1-439d-a1e5-909708c4a2a0-konnectivity-ca\") pod \"konnectivity-agent-bl8kt\" (UID: \"a61e4596-f5a1-439d-a1e5-909708c4a2a0\") " pod="kube-system/konnectivity-agent-bl8kt" Jul 02 19:05:22.789780 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.789332 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-system-cni-dir\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.789780 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.789368 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bcd6dcd7-7601-48bc-a635-b32b28d95a64-host-slash\") pod \"iptables-alerter-kpjfw\" (UID: \"bcd6dcd7-7601-48bc-a635-b32b28d95a64\") " pod="openshift-network-operator/iptables-alerter-kpjfw" Jul 02 19:05:22.790039 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.789836 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a61e4596-f5a1-439d-a1e5-909708c4a2a0-konnectivity-ca\") pod \"konnectivity-agent-bl8kt\" (UID: \"a61e4596-f5a1-439d-a1e5-909708c4a2a0\") " pod="kube-system/konnectivity-agent-bl8kt" Jul 02 19:05:22.790039 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.789927 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-host-var-lib-cni-multus\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.790039 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.789961 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-etc-kubernetes\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.790039 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.789989 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e-host\") pod \"node-ca-45z57\" (UID: \"d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e\") " pod="openshift-image-registry/node-ca-45z57" Jul 02 19:05:22.790039 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790024 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69f9s\" (UniqueName: \"kubernetes.io/projected/d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e-kube-api-access-69f9s\") pod \"node-ca-45z57\" (UID: \"d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e\") " pod="openshift-image-registry/node-ca-45z57" Jul 02 19:05:22.790175 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790051 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-host-var-lib-kubelet\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.790175 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790066 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-etc-kubernetes\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.790175 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790076 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-multus-conf-dir\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.790175 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790102 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-host-slash\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.790175 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790123 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-host-run-netns\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.790175 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790150 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/85a38081-5d7f-444a-837e-f536c6049157-ovnkube-config\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.790333 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790179 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/85a38081-5d7f-444a-837e-f536c6049157-env-overrides\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.790333 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790202 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-host-run-netns\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.790333 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790227 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbfqb\" (UniqueName: \"kubernetes.io/projected/92620d26-b178-4330-a011-98af64b4d43c-kube-api-access-pbfqb\") pod \"network-metrics-daemon-c9b6h\" (UID: \"92620d26-b178-4330-a011-98af64b4d43c\") " pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:22.790333 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790252 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.790333 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790277 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.790333 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790303 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3cf388a6-8721-429f-a2f9-f5f950e40a3b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9mbwg\" (UID: \"3cf388a6-8721-429f-a2f9-f5f950e40a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.790333 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790330 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3cf388a6-8721-429f-a2f9-f5f950e40a3b-sys-fs\") pod \"aws-ebs-csi-driver-node-9mbwg\" (UID: \"3cf388a6-8721-429f-a2f9-f5f950e40a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.790525 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790353 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-log-socket\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.790525 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790303 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e-host\") pod \"node-ca-45z57\" (UID: \"d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e\") " pod="openshift-image-registry/node-ca-45z57" Jul 02 19:05:22.790525 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790417 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvqr2\" (UniqueName: \"kubernetes.io/projected/85a38081-5d7f-444a-837e-f536c6049157-kube-api-access-lvqr2\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.790525 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790433 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3cf388a6-8721-429f-a2f9-f5f950e40a3b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9mbwg\" (UID: \"3cf388a6-8721-429f-a2f9-f5f950e40a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.790525 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790441 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-host-var-lib-cni-bin\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.790525 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790434 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3cf388a6-8721-429f-a2f9-f5f950e40a3b-sys-fs\") pod \"aws-ebs-csi-driver-node-9mbwg\" (UID: \"3cf388a6-8721-429f-a2f9-f5f950e40a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.790525 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790474 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-cni-binary-copy\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.790525 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790503 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6ftx\" (UniqueName: \"kubernetes.io/projected/3cf388a6-8721-429f-a2f9-f5f950e40a3b-kube-api-access-l6ftx\") pod \"aws-ebs-csi-driver-node-9mbwg\" (UID: \"3cf388a6-8721-429f-a2f9-f5f950e40a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.790760 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790531 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-os-release\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.790760 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790598 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/534a1f91-37b3-4ba4-93fc-f872ed2b668f-multus-daemon-config\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.790760 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790626 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret\") pod \"global-pull-secret-syncer-txtgt\" (UID: \"0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef\") " pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:22.790760 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790650 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-lib-modules\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.790760 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790691 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bcd6dcd7-7601-48bc-a635-b32b28d95a64-iptables-alerter-script\") pod \"iptables-alerter-kpjfw\" (UID: \"bcd6dcd7-7601-48bc-a635-b32b28d95a64\") " pod="openshift-network-operator/iptables-alerter-kpjfw" Jul 02 19:05:22.790760 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790715 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-hostroot\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.790760 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790742 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-kubelet-config\") pod \"global-pull-secret-syncer-txtgt\" (UID: \"0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef\") " pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:22.790980 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790809 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-dbus\") pod \"global-pull-secret-syncer-txtgt\" (UID: \"0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef\") " pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:22.790980 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790835 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-var-lib-kubelet\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.790980 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790880 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3cf388a6-8721-429f-a2f9-f5f950e40a3b-etc-selinux\") pod \"aws-ebs-csi-driver-node-9mbwg\" (UID: \"3cf388a6-8721-429f-a2f9-f5f950e40a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.790980 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790906 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zpq7\" (UniqueName: \"kubernetes.io/projected/534a1f91-37b3-4ba4-93fc-f872ed2b668f-kube-api-access-5zpq7\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.790980 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790933 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-run\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.790980 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790970 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.791141 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.790990 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/719b7248-85a7-4687-a8be-e806e338165f-tmp\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.791141 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.791016 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-etc-openvswitch\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.791141 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.791043 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-multus-cni-dir\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.791141 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.791071 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-etc-sysconfig\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.791141 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.791095 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-etc-sysctl-conf\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.791141 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.791121 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.791300 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.791215 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-cni-binary-copy\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.791300 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.791256 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3cf388a6-8721-429f-a2f9-f5f950e40a3b-socket-dir\") pod \"aws-ebs-csi-driver-node-9mbwg\" (UID: \"3cf388a6-8721-429f-a2f9-f5f950e40a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.791817 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.791431 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.791817 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.791492 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-etc-sysconfig\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.791817 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.791492 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-lib-modules\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.791817 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.791514 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-run\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.791817 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.791555 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-etc-sysctl-conf\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.791817 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.791579 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/719b7248-85a7-4687-a8be-e806e338165f-var-lib-kubelet\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.791817 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.791578 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.791817 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.791604 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3cf388a6-8721-429f-a2f9-f5f950e40a3b-device-dir\") pod \"aws-ebs-csi-driver-node-9mbwg\" (UID: \"3cf388a6-8721-429f-a2f9-f5f950e40a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.791817 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.791622 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3cf388a6-8721-429f-a2f9-f5f950e40a3b-socket-dir\") pod \"aws-ebs-csi-driver-node-9mbwg\" (UID: \"3cf388a6-8721-429f-a2f9-f5f950e40a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.791817 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.791648 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3cf388a6-8721-429f-a2f9-f5f950e40a3b-etc-selinux\") pod \"aws-ebs-csi-driver-node-9mbwg\" (UID: \"3cf388a6-8721-429f-a2f9-f5f950e40a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.791817 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.791715 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3cf388a6-8721-429f-a2f9-f5f950e40a3b-device-dir\") pod \"aws-ebs-csi-driver-node-9mbwg\" (UID: \"3cf388a6-8721-429f-a2f9-f5f950e40a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.792295 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.791837 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bcd6dcd7-7601-48bc-a635-b32b28d95a64-iptables-alerter-script\") pod \"iptables-alerter-kpjfw\" (UID: \"bcd6dcd7-7601-48bc-a635-b32b28d95a64\") " pod="openshift-network-operator/iptables-alerter-kpjfw" Jul 02 19:05:22.792295 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.792053 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 19:05:22.792295 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.792282 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/719b7248-85a7-4687-a8be-e806e338165f-etc-tuned\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.793473 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.793452 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a61e4596-f5a1-439d-a1e5-909708c4a2a0-agent-certs\") pod \"konnectivity-agent-bl8kt\" (UID: \"a61e4596-f5a1-439d-a1e5-909708c4a2a0\") " pod="kube-system/konnectivity-agent-bl8kt" Jul 02 19:05:22.793562 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.793548 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/719b7248-85a7-4687-a8be-e806e338165f-tmp\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.799589 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:22.798463 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:05:22.799589 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:22.798503 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:05:22.799589 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:22.798516 2566 projected.go:196] Error preparing data for projected volume kube-api-access-xhqhb for pod openshift-network-diagnostics/network-check-target-hssx4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:05:22.799589 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:22.798587 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb podName:4d87225d-39f9-40d0-8ae7-5fa11099c16c nodeName:}" failed. No retries permitted until 2026-07-02 19:05:23.29857071 +0000 UTC m=+3.225579423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xhqhb" (UniqueName: "kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb") pod "network-check-target-hssx4" (UID: "4d87225d-39f9-40d0-8ae7-5fa11099c16c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:05:22.803356 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.803332 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69f9s\" (UniqueName: \"kubernetes.io/projected/d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e-kube-api-access-69f9s\") pod \"node-ca-45z57\" (UID: \"d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e\") " pod="openshift-image-registry/node-ca-45z57" Jul 02 19:05:22.803687 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.803650 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n9p2\" (UniqueName: \"kubernetes.io/projected/8e4ef969-3b7b-41a4-865b-f15c27d2cd69-kube-api-access-9n9p2\") pod \"multus-additional-cni-plugins-6nnrq\" (UID: \"8e4ef969-3b7b-41a4-865b-f15c27d2cd69\") " pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:22.803803 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.803781 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9p58\" (UniqueName: \"kubernetes.io/projected/719b7248-85a7-4687-a8be-e806e338165f-kube-api-access-n9p58\") pod \"tuned-7hnbs\" (UID: \"719b7248-85a7-4687-a8be-e806e338165f\") " pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:22.803906 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.803886 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbfqb\" (UniqueName: \"kubernetes.io/projected/92620d26-b178-4330-a011-98af64b4d43c-kube-api-access-pbfqb\") pod \"network-metrics-daemon-c9b6h\" (UID: \"92620d26-b178-4330-a011-98af64b4d43c\") " pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:22.804018 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.803999 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhzv5\" (UniqueName: \"kubernetes.io/projected/bcd6dcd7-7601-48bc-a635-b32b28d95a64-kube-api-access-jhzv5\") pod \"iptables-alerter-kpjfw\" (UID: \"bcd6dcd7-7601-48bc-a635-b32b28d95a64\") " pod="openshift-network-operator/iptables-alerter-kpjfw" Jul 02 19:05:22.804217 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.804199 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfx2z\" (UniqueName: \"kubernetes.io/projected/a04cbb97-93a7-4d61-bdf9-deba1968e7c0-kube-api-access-wfx2z\") pod \"node-resolver-hbf5k\" (UID: \"a04cbb97-93a7-4d61-bdf9-deba1968e7c0\") " pod="openshift-dns/node-resolver-hbf5k" Jul 02 19:05:22.804248 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.804220 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6ftx\" (UniqueName: \"kubernetes.io/projected/3cf388a6-8721-429f-a2f9-f5f950e40a3b-kube-api-access-l6ftx\") pod \"aws-ebs-csi-driver-node-9mbwg\" (UID: \"3cf388a6-8721-429f-a2f9-f5f950e40a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.842048 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.841990 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-205.ec2.internal" event={"ID":"98feeb3b547af7c850dfc4e76f035021","Type":"ContainerStarted","Data":"12d2a40a896940e67ad0dd6a4c8bc033833ce29daaaa9381a7a76e51c33eaed4"} Jul 02 19:05:22.843222 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.843199 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-205.ec2.internal" event={"ID":"443b95f9a11c785c66a6608d9d6fba98","Type":"ContainerStarted","Data":"733017917baaba2552b2f6231c8cd4568b0aea6d1a6b012044563624158fc7ae"} Jul 02 19:05:22.891948 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.891864 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zpq7\" (UniqueName: \"kubernetes.io/projected/534a1f91-37b3-4ba4-93fc-f872ed2b668f-kube-api-access-5zpq7\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.891948 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.891901 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-etc-openvswitch\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.891948 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.891920 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-multus-cni-dir\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.892196 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.891987 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-etc-openvswitch\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892196 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892030 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-systemd-units\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892196 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892055 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-host-cni-netd\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892196 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892078 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-host-cni-bin\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892196 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892101 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/85a38081-5d7f-444a-837e-f536c6049157-ovnkube-script-lib\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892196 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892129 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-multus-cni-dir\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.892196 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892171 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-systemd-units\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892467 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892215 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/534a1f91-37b3-4ba4-93fc-f872ed2b668f-cni-binary-copy\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.892467 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892242 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-etc-kubernetes\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.892467 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892281 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-var-lib-openvswitch\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892467 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892303 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892467 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892334 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-host-kubelet\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892467 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892366 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-host-run-ovn-kubernetes\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892467 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892393 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-run-systemd\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892467 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892412 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-run-openvswitch\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892467 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892433 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/85a38081-5d7f-444a-837e-f536c6049157-ovn-node-metrics-cert\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892467 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892449 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-cnibin\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.892467 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892463 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892908 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892474 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-multus-socket-dir-parent\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.892908 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892491 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-host-run-multus-certs\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.892908 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892515 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-host-run-k8s-cni-cncf-io\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.892908 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892541 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-run-ovn\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892908 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892560 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-node-log\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892908 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892605 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-system-cni-dir\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.892908 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892623 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-host-var-lib-cni-multus\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.892908 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892640 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-host-var-lib-kubelet\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.892908 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892655 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-multus-conf-dir\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.892908 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892668 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-host-slash\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892908 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892709 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-host-run-netns\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892908 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892743 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/85a38081-5d7f-444a-837e-f536c6049157-ovnkube-config\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892908 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892780 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/85a38081-5d7f-444a-837e-f536c6049157-env-overrides\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892908 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892806 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-host-run-netns\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.892908 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892826 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-log-socket\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892908 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892832 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/85a38081-5d7f-444a-837e-f536c6049157-ovnkube-script-lib\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892908 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892880 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-host-run-ovn-kubernetes\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.892908 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892910 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-run-systemd\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.893533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892942 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-run-openvswitch\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.893533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893028 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/534a1f91-37b3-4ba4-93fc-f872ed2b668f-cni-binary-copy\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.893533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893086 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-etc-kubernetes\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.893533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893096 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-host-cni-netd\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.893533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893126 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-var-lib-openvswitch\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.893533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893157 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-host-kubelet\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.893533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.892840 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvqr2\" (UniqueName: \"kubernetes.io/projected/85a38081-5d7f-444a-837e-f536c6049157-kube-api-access-lvqr2\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.893533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893173 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-host-var-lib-cni-multus\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.893533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893209 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-host-var-lib-cni-bin\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.893533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893225 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-cnibin\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.893533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893236 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-os-release\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.893533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893252 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/534a1f91-37b3-4ba4-93fc-f872ed2b668f-multus-daemon-config\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.893533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893270 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret\") pod \"global-pull-secret-syncer-txtgt\" (UID: \"0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef\") " pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:22.893533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893271 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-multus-socket-dir-parent\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.893533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893287 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-hostroot\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.893533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893304 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-kubelet-config\") pod \"global-pull-secret-syncer-txtgt\" (UID: \"0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef\") " pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:22.893533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893311 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-host-run-multus-certs\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.893533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893326 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-dbus\") pod \"global-pull-secret-syncer-txtgt\" (UID: \"0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef\") " pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:22.894362 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893354 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-host-run-k8s-cni-cncf-io\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.894362 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893396 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-run-ovn\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.894362 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893408 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-dbus\") pod \"global-pull-secret-syncer-txtgt\" (UID: \"0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef\") " pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:22.894362 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893446 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-system-cni-dir\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.894362 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893450 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-host-cni-bin\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.894362 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893474 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-node-log\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.894362 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893510 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-host-var-lib-cni-bin\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.894362 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:22.893524 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:05:22.894362 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893571 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-os-release\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.894362 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:22.893584 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret podName:0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef nodeName:}" failed. No retries permitted until 2026-07-02 19:05:23.393567693 +0000 UTC m=+3.320576385 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret") pod "global-pull-secret-syncer-txtgt" (UID: "0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:05:22.894362 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893602 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-host-var-lib-kubelet\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.894362 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.893625 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-kubelet-config\") pod \"global-pull-secret-syncer-txtgt\" (UID: \"0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef\") " pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:22.894362 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.894080 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/85a38081-5d7f-444a-837e-f536c6049157-env-overrides\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.894362 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.894082 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/534a1f91-37b3-4ba4-93fc-f872ed2b668f-multus-daemon-config\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.894362 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.894118 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/85a38081-5d7f-444a-837e-f536c6049157-ovnkube-config\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.894362 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.894154 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-host-run-netns\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.894362 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.894172 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-hostroot\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.894362 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.894200 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-log-socket\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.895046 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.894242 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-host-slash\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.895046 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.894250 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85a38081-5d7f-444a-837e-f536c6049157-host-run-netns\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.895046 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.894275 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/534a1f91-37b3-4ba4-93fc-f872ed2b668f-multus-conf-dir\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.895468 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.895448 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/85a38081-5d7f-444a-837e-f536c6049157-ovn-node-metrics-cert\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.904109 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.904085 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zpq7\" (UniqueName: \"kubernetes.io/projected/534a1f91-37b3-4ba4-93fc-f872ed2b668f-kube-api-access-5zpq7\") pod \"multus-z8v4v\" (UID: \"534a1f91-37b3-4ba4-93fc-f872ed2b668f\") " pod="openshift-multus/multus-z8v4v" Jul 02 19:05:22.905063 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.905042 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvqr2\" (UniqueName: \"kubernetes.io/projected/85a38081-5d7f-444a-837e-f536c6049157-kube-api-access-lvqr2\") pod \"ovnkube-node-29hfm\" (UID: \"85a38081-5d7f-444a-837e-f536c6049157\") " pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.975150 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.975114 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" Jul 02 19:05:22.984288 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.984264 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hbf5k" Jul 02 19:05:22.994091 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.994067 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:22.999819 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:22.999798 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-45z57" Jul 02 19:05:23.008346 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:23.008324 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kpjfw" Jul 02 19:05:23.014922 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:23.014891 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bl8kt" Jul 02 19:05:23.022590 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:23.022571 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" Jul 02 19:05:23.030321 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:23.030304 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6nnrq" Jul 02 19:05:23.035962 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:23.035942 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z8v4v" Jul 02 19:05:23.296140 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:23.296064 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs\") pod \"network-metrics-daemon-c9b6h\" (UID: \"92620d26-b178-4330-a011-98af64b4d43c\") " pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:23.296285 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:23.296191 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:05:23.296285 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:23.296248 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs podName:92620d26-b178-4330-a011-98af64b4d43c nodeName:}" failed. No retries permitted until 2026-07-02 19:05:24.29623262 +0000 UTC m=+4.223241316 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs") pod "network-metrics-daemon-c9b6h" (UID: "92620d26-b178-4330-a011-98af64b4d43c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:05:23.397084 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:23.397049 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhqhb\" (UniqueName: \"kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb\") pod \"network-check-target-hssx4\" (UID: \"4d87225d-39f9-40d0-8ae7-5fa11099c16c\") " pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:23.397275 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:23.397123 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret\") pod \"global-pull-secret-syncer-txtgt\" (UID: \"0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef\") " pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:23.397275 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:23.397221 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:05:23.397275 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:23.397239 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:05:23.397275 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:23.397258 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:05:23.397275 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:23.397267 2566 projected.go:196] Error preparing data for projected volume kube-api-access-xhqhb for pod openshift-network-diagnostics/network-check-target-hssx4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:05:23.397519 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:23.397289 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret podName:0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef nodeName:}" failed. No retries permitted until 2026-07-02 19:05:24.397272572 +0000 UTC m=+4.324281267 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret") pod "global-pull-secret-syncer-txtgt" (UID: "0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:05:23.397519 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:23.397317 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb podName:4d87225d-39f9-40d0-8ae7-5fa11099c16c nodeName:}" failed. No retries permitted until 2026-07-02 19:05:24.397302877 +0000 UTC m=+4.324311590 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhqhb" (UniqueName: "kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb") pod "network-check-target-hssx4" (UID: "4d87225d-39f9-40d0-8ae7-5fa11099c16c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:05:23.736736 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:23.736588 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 19:00:22 +0000 UTC" deadline="2028-02-19 05:23:04.234340228 +0000 UTC" Jul 02 19:05:23.736736 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:23.736619 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14314h17m40.497724975s" Jul 02 19:05:23.752023 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:23.752000 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod719b7248_85a7_4687_a8be_e806e338165f.slice/crio-99cd1829e2f50847e7c0997128e907e54348adfcc67296b0b7c8ceee282acdf0 WatchSource:0}: Error finding container 99cd1829e2f50847e7c0997128e907e54348adfcc67296b0b7c8ceee282acdf0: Status 404 returned error can't find the container with id 99cd1829e2f50847e7c0997128e907e54348adfcc67296b0b7c8ceee282acdf0 Jul 02 19:05:23.754393 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:23.754372 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e4ef969_3b7b_41a4_865b_f15c27d2cd69.slice/crio-77387056ac9aa0e0756f15db0201cdff8376a1b1b09a46d8d7ee2584b0ea0853 WatchSource:0}: Error finding container 77387056ac9aa0e0756f15db0201cdff8376a1b1b09a46d8d7ee2584b0ea0853: Status 404 returned error can't find the container with id 77387056ac9aa0e0756f15db0201cdff8376a1b1b09a46d8d7ee2584b0ea0853 Jul 02 19:05:23.755742 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:23.755234 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcd6dcd7_7601_48bc_a635_b32b28d95a64.slice/crio-fd00f8376096ae3d33c1e2eb51d70032dd85c7296aaf82f08b8c17561bdb7ee0 WatchSource:0}: Error finding container fd00f8376096ae3d33c1e2eb51d70032dd85c7296aaf82f08b8c17561bdb7ee0: Status 404 returned error can't find the container with id fd00f8376096ae3d33c1e2eb51d70032dd85c7296aaf82f08b8c17561bdb7ee0 Jul 02 19:05:23.758326 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:23.758296 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cf388a6_8721_429f_a2f9_f5f950e40a3b.slice/crio-a71425c59f627b2cdc319add48949f2fbfcbface2395d66d94902f1722c6bae0 WatchSource:0}: Error finding container a71425c59f627b2cdc319add48949f2fbfcbface2395d66d94902f1722c6bae0: Status 404 returned error can't find the container with id a71425c59f627b2cdc319add48949f2fbfcbface2395d66d94902f1722c6bae0 Jul 02 19:05:23.759458 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:23.759433 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod534a1f91_37b3_4ba4_93fc_f872ed2b668f.slice/crio-ece7024a1ffa3df7227dc90b293058364abadeab4c6fc363a41c91f87c6215c2 WatchSource:0}: Error finding container ece7024a1ffa3df7227dc90b293058364abadeab4c6fc363a41c91f87c6215c2: Status 404 returned error can't find the container with id ece7024a1ffa3df7227dc90b293058364abadeab4c6fc363a41c91f87c6215c2 Jul 02 19:05:23.760223 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:23.760090 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85a38081_5d7f_444a_837e_f536c6049157.slice/crio-30df194cdc4f9fc910b73d75ea2120364073ab9f7777ebb7a99008c044dc5cda WatchSource:0}: Error finding container 30df194cdc4f9fc910b73d75ea2120364073ab9f7777ebb7a99008c044dc5cda: Status 404 returned error can't find the container with id 30df194cdc4f9fc910b73d75ea2120364073ab9f7777ebb7a99008c044dc5cda Jul 02 19:05:23.765930 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:23.765638 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3b9eda7_ed8d_4a5b_9f6d_7c13e252fe5e.slice/crio-cfa9419aa69ea49b5391b4a216a4be8812acdcecb4a99986760bbcf8f93b7828 WatchSource:0}: Error finding container cfa9419aa69ea49b5391b4a216a4be8812acdcecb4a99986760bbcf8f93b7828: Status 404 returned error can't find the container with id cfa9419aa69ea49b5391b4a216a4be8812acdcecb4a99986760bbcf8f93b7828 Jul 02 19:05:23.838723 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:23.838556 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:23.838823 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:23.838808 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9b6h" podUID="92620d26-b178-4330-a011-98af64b4d43c" Jul 02 19:05:23.845834 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:23.845805 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" event={"ID":"85a38081-5d7f-444a-837e-f536c6049157","Type":"ContainerStarted","Data":"30df194cdc4f9fc910b73d75ea2120364073ab9f7777ebb7a99008c044dc5cda"} Jul 02 19:05:23.846982 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:23.846958 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z8v4v" event={"ID":"534a1f91-37b3-4ba4-93fc-f872ed2b668f","Type":"ContainerStarted","Data":"ece7024a1ffa3df7227dc90b293058364abadeab4c6fc363a41c91f87c6215c2"} Jul 02 19:05:23.849381 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:23.849276 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6nnrq" event={"ID":"8e4ef969-3b7b-41a4-865b-f15c27d2cd69","Type":"ContainerStarted","Data":"77387056ac9aa0e0756f15db0201cdff8376a1b1b09a46d8d7ee2584b0ea0853"} Jul 02 19:05:23.850949 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:23.850910 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" event={"ID":"719b7248-85a7-4687-a8be-e806e338165f","Type":"ContainerStarted","Data":"99cd1829e2f50847e7c0997128e907e54348adfcc67296b0b7c8ceee282acdf0"} Jul 02 19:05:23.853003 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:23.852972 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-205.ec2.internal" event={"ID":"443b95f9a11c785c66a6608d9d6fba98","Type":"ContainerStarted","Data":"7bba398909bd712e7846d76dced7371b02c945cb8ed2b2fa054fd23eaff1023d"} Jul 02 19:05:23.854028 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:23.854007 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-45z57" event={"ID":"d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e","Type":"ContainerStarted","Data":"cfa9419aa69ea49b5391b4a216a4be8812acdcecb4a99986760bbcf8f93b7828"} Jul 02 19:05:23.854939 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:23.854922 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kpjfw" event={"ID":"bcd6dcd7-7601-48bc-a635-b32b28d95a64","Type":"ContainerStarted","Data":"fd00f8376096ae3d33c1e2eb51d70032dd85c7296aaf82f08b8c17561bdb7ee0"} Jul 02 19:05:23.856068 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:23.856014 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" event={"ID":"3cf388a6-8721-429f-a2f9-f5f950e40a3b","Type":"ContainerStarted","Data":"a71425c59f627b2cdc319add48949f2fbfcbface2395d66d94902f1722c6bae0"} Jul 02 19:05:23.857122 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:23.857102 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hbf5k" event={"ID":"a04cbb97-93a7-4d61-bdf9-deba1968e7c0","Type":"ContainerStarted","Data":"a39c7a6ac31ff2de3638fa8335adaf39df327797a4cc731bafa11a6186dcad83"} Jul 02 19:05:23.858097 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:23.858075 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bl8kt" event={"ID":"a61e4596-f5a1-439d-a1e5-909708c4a2a0","Type":"ContainerStarted","Data":"ae716ae23624e3dafd72d661bf91aa771a88ab3895364ce5e0fc33769da0a778"} Jul 02 19:05:24.303429 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:24.303337 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs\") pod \"network-metrics-daemon-c9b6h\" (UID: \"92620d26-b178-4330-a011-98af64b4d43c\") " pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:24.303583 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:24.303484 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:05:24.303583 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:24.303546 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs podName:92620d26-b178-4330-a011-98af64b4d43c nodeName:}" failed. No retries permitted until 2026-07-02 19:05:26.303528999 +0000 UTC m=+6.230537703 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs") pod "network-metrics-daemon-c9b6h" (UID: "92620d26-b178-4330-a011-98af64b4d43c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:05:24.403966 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:24.403928 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret\") pod \"global-pull-secret-syncer-txtgt\" (UID: \"0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef\") " pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:24.404126 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:24.403991 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhqhb\" (UniqueName: \"kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb\") pod \"network-check-target-hssx4\" (UID: \"4d87225d-39f9-40d0-8ae7-5fa11099c16c\") " pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:24.404183 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:24.404167 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:05:24.404229 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:24.404185 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:05:24.404229 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:24.404198 2566 projected.go:196] Error preparing data for projected volume kube-api-access-xhqhb for pod openshift-network-diagnostics/network-check-target-hssx4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:05:24.404341 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:24.404259 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb podName:4d87225d-39f9-40d0-8ae7-5fa11099c16c nodeName:}" failed. No retries permitted until 2026-07-02 19:05:26.404240745 +0000 UTC m=+6.331249457 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhqhb" (UniqueName: "kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb") pod "network-check-target-hssx4" (UID: "4d87225d-39f9-40d0-8ae7-5fa11099c16c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:05:24.404691 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:24.404650 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:05:24.404805 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:24.404722 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret podName:0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef nodeName:}" failed. No retries permitted until 2026-07-02 19:05:26.404706842 +0000 UTC m=+6.331715533 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret") pod "global-pull-secret-syncer-txtgt" (UID: "0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:05:24.737318 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:24.737254 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 19:00:22 +0000 UTC" deadline="2028-03-10 05:29:49.357607024 +0000 UTC" Jul 02 19:05:24.737318 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:24.737317 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14794h24m24.620294863s" Jul 02 19:05:24.840338 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:24.840120 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:24.840338 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:24.840249 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txtgt" podUID="0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef" Jul 02 19:05:24.840338 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:24.840327 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:24.840611 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:24.840393 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hssx4" podUID="4d87225d-39f9-40d0-8ae7-5fa11099c16c" Jul 02 19:05:24.876902 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:24.875713 2566 generic.go:356] "Generic (PLEG): container finished" podID="98feeb3b547af7c850dfc4e76f035021" containerID="65cbdcbc01301093bfce3b9b075d9f93060ac1eabf3c13d5d268ca25806eab83" exitCode=0 Jul 02 19:05:24.876902 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:24.876651 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-205.ec2.internal" event={"ID":"98feeb3b547af7c850dfc4e76f035021","Type":"ContainerDied","Data":"65cbdcbc01301093bfce3b9b075d9f93060ac1eabf3c13d5d268ca25806eab83"} Jul 02 19:05:24.891099 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:24.891037 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-205.ec2.internal" podStartSLOduration=2.891018258 podStartE2EDuration="2.891018258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:23.86825617 +0000 UTC m=+3.795264881" watchObservedRunningTime="2026-07-02 19:05:24.891018258 +0000 UTC m=+4.818026972" Jul 02 19:05:25.838488 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:25.838452 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:25.838994 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:25.838601 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9b6h" podUID="92620d26-b178-4330-a011-98af64b4d43c" Jul 02 19:05:25.887493 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:25.887416 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-205.ec2.internal" event={"ID":"98feeb3b547af7c850dfc4e76f035021","Type":"ContainerStarted","Data":"98ede64ebb0a50b7f63b55d57031aeabae66b438eee9361df2c2c89bb0f9e064"} Jul 02 19:05:26.321329 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:26.321239 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs\") pod \"network-metrics-daemon-c9b6h\" (UID: \"92620d26-b178-4330-a011-98af64b4d43c\") " pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:26.321542 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:26.321403 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:05:26.321542 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:26.321482 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs podName:92620d26-b178-4330-a011-98af64b4d43c nodeName:}" failed. No retries permitted until 2026-07-02 19:05:30.321463147 +0000 UTC m=+10.248471843 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs") pod "network-metrics-daemon-c9b6h" (UID: "92620d26-b178-4330-a011-98af64b4d43c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:05:26.422016 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:26.421968 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret\") pod \"global-pull-secret-syncer-txtgt\" (UID: \"0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef\") " pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:26.422206 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:26.422047 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhqhb\" (UniqueName: \"kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb\") pod \"network-check-target-hssx4\" (UID: \"4d87225d-39f9-40d0-8ae7-5fa11099c16c\") " pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:26.422269 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:26.422226 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:05:26.422269 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:26.422248 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:05:26.422269 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:26.422261 2566 projected.go:196] Error preparing data for projected volume kube-api-access-xhqhb for pod openshift-network-diagnostics/network-check-target-hssx4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:05:26.422422 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:26.422321 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb podName:4d87225d-39f9-40d0-8ae7-5fa11099c16c nodeName:}" failed. No retries permitted until 2026-07-02 19:05:30.422301959 +0000 UTC m=+10.349310653 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhqhb" (UniqueName: "kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb") pod "network-check-target-hssx4" (UID: "4d87225d-39f9-40d0-8ae7-5fa11099c16c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:05:26.422422 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:26.422397 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:05:26.422531 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:26.422433 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret podName:0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef nodeName:}" failed. No retries permitted until 2026-07-02 19:05:30.422420729 +0000 UTC m=+10.349429422 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret") pod "global-pull-secret-syncer-txtgt" (UID: "0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:05:26.838412 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:26.838335 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:26.839042 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:26.839000 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txtgt" podUID="0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef" Jul 02 19:05:26.839439 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:26.839034 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:26.839439 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:26.839305 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hssx4" podUID="4d87225d-39f9-40d0-8ae7-5fa11099c16c" Jul 02 19:05:27.837880 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:27.837848 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:27.838082 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:27.837998 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9b6h" podUID="92620d26-b178-4330-a011-98af64b4d43c" Jul 02 19:05:28.838516 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:28.838478 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:28.838516 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:28.838517 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:28.839067 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:28.838621 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txtgt" podUID="0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef" Jul 02 19:05:28.839067 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:28.838754 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hssx4" podUID="4d87225d-39f9-40d0-8ae7-5fa11099c16c" Jul 02 19:05:29.838551 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:29.838472 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:29.839050 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:29.838633 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9b6h" podUID="92620d26-b178-4330-a011-98af64b4d43c" Jul 02 19:05:30.357195 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:30.357154 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs\") pod \"network-metrics-daemon-c9b6h\" (UID: \"92620d26-b178-4330-a011-98af64b4d43c\") " pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:30.357416 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:30.357396 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:05:30.357488 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:30.357478 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs podName:92620d26-b178-4330-a011-98af64b4d43c nodeName:}" failed. No retries permitted until 2026-07-02 19:05:38.357455338 +0000 UTC m=+18.284464048 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs") pod "network-metrics-daemon-c9b6h" (UID: "92620d26-b178-4330-a011-98af64b4d43c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:05:30.458026 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:30.457974 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret\") pod \"global-pull-secret-syncer-txtgt\" (UID: \"0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef\") " pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:30.458210 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:30.458102 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhqhb\" (UniqueName: \"kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb\") pod \"network-check-target-hssx4\" (UID: \"4d87225d-39f9-40d0-8ae7-5fa11099c16c\") " pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:30.458288 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:30.458272 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:05:30.458345 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:30.458298 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:05:30.458345 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:30.458314 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:05:30.458345 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:30.458326 2566 projected.go:196] Error preparing data for projected volume kube-api-access-xhqhb for pod openshift-network-diagnostics/network-check-target-hssx4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:05:30.458495 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:30.458364 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret podName:0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef nodeName:}" failed. No retries permitted until 2026-07-02 19:05:38.45834394 +0000 UTC m=+18.385352650 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret") pod "global-pull-secret-syncer-txtgt" (UID: "0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:05:30.458495 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:30.458383 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb podName:4d87225d-39f9-40d0-8ae7-5fa11099c16c nodeName:}" failed. No retries permitted until 2026-07-02 19:05:38.458373815 +0000 UTC m=+18.385382512 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhqhb" (UniqueName: "kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb") pod "network-check-target-hssx4" (UID: "4d87225d-39f9-40d0-8ae7-5fa11099c16c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:05:30.839932 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:30.839851 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:30.840377 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:30.839971 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txtgt" podUID="0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef" Jul 02 19:05:30.840437 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:30.840393 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:30.840505 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:30.840485 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hssx4" podUID="4d87225d-39f9-40d0-8ae7-5fa11099c16c" Jul 02 19:05:31.838404 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:31.838328 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:31.838589 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:31.838473 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9b6h" podUID="92620d26-b178-4330-a011-98af64b4d43c" Jul 02 19:05:32.838806 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:32.838772 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:32.838806 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:32.838801 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:32.839277 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:32.838907 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txtgt" podUID="0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef" Jul 02 19:05:32.839277 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:32.839081 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hssx4" podUID="4d87225d-39f9-40d0-8ae7-5fa11099c16c" Jul 02 19:05:33.837951 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:33.837912 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:33.838161 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:33.838065 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9b6h" podUID="92620d26-b178-4330-a011-98af64b4d43c" Jul 02 19:05:34.838102 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:34.838062 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:34.838102 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:34.838086 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:34.838596 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:34.838211 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txtgt" podUID="0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef" Jul 02 19:05:34.838596 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:34.838232 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hssx4" podUID="4d87225d-39f9-40d0-8ae7-5fa11099c16c" Jul 02 19:05:35.838051 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:35.838016 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:35.838225 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:35.838156 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9b6h" podUID="92620d26-b178-4330-a011-98af64b4d43c" Jul 02 19:05:36.838781 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:36.838747 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:36.838781 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:36.838782 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:36.839259 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:36.838888 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txtgt" podUID="0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef" Jul 02 19:05:36.839259 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:36.839037 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hssx4" podUID="4d87225d-39f9-40d0-8ae7-5fa11099c16c" Jul 02 19:05:37.837734 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:37.837699 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:37.837923 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:37.837834 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9b6h" podUID="92620d26-b178-4330-a011-98af64b4d43c" Jul 02 19:05:38.419000 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:38.418957 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs\") pod \"network-metrics-daemon-c9b6h\" (UID: \"92620d26-b178-4330-a011-98af64b4d43c\") " pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:38.419462 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:38.419128 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:05:38.419462 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:38.419214 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs podName:92620d26-b178-4330-a011-98af64b4d43c nodeName:}" failed. No retries permitted until 2026-07-02 19:05:54.419191994 +0000 UTC m=+34.346200705 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs") pod "network-metrics-daemon-c9b6h" (UID: "92620d26-b178-4330-a011-98af64b4d43c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:05:38.519514 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:38.519478 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhqhb\" (UniqueName: \"kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb\") pod \"network-check-target-hssx4\" (UID: \"4d87225d-39f9-40d0-8ae7-5fa11099c16c\") " pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:38.519670 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:38.519552 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret\") pod \"global-pull-secret-syncer-txtgt\" (UID: \"0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef\") " pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:38.519670 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:38.519638 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:05:38.519670 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:38.519648 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:05:38.519670 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:38.519666 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:05:38.519865 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:38.519699 2566 projected.go:196] Error preparing data for projected volume kube-api-access-xhqhb for pod openshift-network-diagnostics/network-check-target-hssx4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:05:38.519865 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:38.519719 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret podName:0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef nodeName:}" failed. No retries permitted until 2026-07-02 19:05:54.519699317 +0000 UTC m=+34.446708007 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret") pod "global-pull-secret-syncer-txtgt" (UID: "0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:05:38.519865 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:38.519739 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb podName:4d87225d-39f9-40d0-8ae7-5fa11099c16c nodeName:}" failed. No retries permitted until 2026-07-02 19:05:54.519726593 +0000 UTC m=+34.446735297 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhqhb" (UniqueName: "kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb") pod "network-check-target-hssx4" (UID: "4d87225d-39f9-40d0-8ae7-5fa11099c16c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:05:38.838417 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:38.838336 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:38.838566 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:38.838456 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txtgt" podUID="0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef" Jul 02 19:05:38.838566 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:38.838519 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:38.838710 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:38.838622 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hssx4" podUID="4d87225d-39f9-40d0-8ae7-5fa11099c16c" Jul 02 19:05:39.838716 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:39.838669 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:39.839186 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:39.838816 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9b6h" podUID="92620d26-b178-4330-a011-98af64b4d43c" Jul 02 19:05:40.839332 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:40.839275 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:40.839817 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:40.839409 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txtgt" podUID="0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef" Jul 02 19:05:40.839817 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:40.839464 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:40.839817 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:40.839574 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hssx4" podUID="4d87225d-39f9-40d0-8ae7-5fa11099c16c" Jul 02 19:05:41.838499 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:41.838463 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:41.838715 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:41.838609 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9b6h" podUID="92620d26-b178-4330-a011-98af64b4d43c" Jul 02 19:05:42.839102 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:42.838836 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:42.839780 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:42.839181 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txtgt" podUID="0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef" Jul 02 19:05:42.839780 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:42.838979 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:42.839780 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:42.839275 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hssx4" podUID="4d87225d-39f9-40d0-8ae7-5fa11099c16c" Jul 02 19:05:42.923749 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:42.923504 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bl8kt" event={"ID":"a61e4596-f5a1-439d-a1e5-909708c4a2a0","Type":"ContainerStarted","Data":"ecc1b38c038743928e02fa946bde40be3fd546dd01235911ad859f71049448a2"} Jul 02 19:05:42.924968 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:42.924943 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" event={"ID":"85a38081-5d7f-444a-837e-f536c6049157","Type":"ContainerStarted","Data":"34e7d33c9038ef534a43a952d6aea5f392a1176c995bb09efe551488bdab7f00"} Jul 02 19:05:42.926284 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:42.926263 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z8v4v" event={"ID":"534a1f91-37b3-4ba4-93fc-f872ed2b668f","Type":"ContainerStarted","Data":"9072da4ca74d1460e4cd93561608cc7e266395ce45f3fa222ebca3367d07aa92"} Jul 02 19:05:42.927713 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:42.927692 2566 generic.go:356] "Generic (PLEG): container finished" podID="8e4ef969-3b7b-41a4-865b-f15c27d2cd69" containerID="555e09e437378f1857d5b852d448d6e50ec5c187f66aafbfae9bc728b21d932f" exitCode=0 Jul 02 19:05:42.927807 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:42.927754 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6nnrq" event={"ID":"8e4ef969-3b7b-41a4-865b-f15c27d2cd69","Type":"ContainerDied","Data":"555e09e437378f1857d5b852d448d6e50ec5c187f66aafbfae9bc728b21d932f"} Jul 02 19:05:42.931227 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:42.931205 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" event={"ID":"719b7248-85a7-4687-a8be-e806e338165f","Type":"ContainerStarted","Data":"74d7d1cbda042f904fb77b0c476819289e33969327befa18edf3299331e794bc"} Jul 02 19:05:42.932658 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:42.932619 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-45z57" event={"ID":"d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e","Type":"ContainerStarted","Data":"822547a6ad8635837d7ee2f0712d878d6e3771ab9e653ce7f93a5723105dee08"} Jul 02 19:05:42.934112 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:42.934090 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" event={"ID":"3cf388a6-8721-429f-a2f9-f5f950e40a3b","Type":"ContainerStarted","Data":"747638b6e5fc3c37f56e0247b120fd6ae2ef815bf65a16b53ec759397ce5fd94"} Jul 02 19:05:42.935459 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:42.935440 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hbf5k" event={"ID":"a04cbb97-93a7-4d61-bdf9-deba1968e7c0","Type":"ContainerStarted","Data":"0d0dc3a310a87201c71ca1a617e91c9f8827844556195ca1fbf5a82d8143c819"} Jul 02 19:05:42.940913 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:42.940869 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-205.ec2.internal" podStartSLOduration=20.94085167 podStartE2EDuration="20.94085167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:25.903398319 +0000 UTC m=+5.830407033" watchObservedRunningTime="2026-07-02 19:05:42.94085167 +0000 UTC m=+22.867860383" Jul 02 19:05:42.941270 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:42.941232 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bl8kt" podStartSLOduration=12.516242542 podStartE2EDuration="22.941222417s" totalImagesPullingTime="10.424979875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:42.940554056 +0000 UTC m=+22.867562779" watchObservedRunningTime="2026-07-02 19:05:42.941222417 +0000 UTC m=+22.868231130" Jul 02 19:05:42.957191 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:42.957152 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7hnbs" podStartSLOduration=4.135013438 podStartE2EDuration="22.957138726s" totalImagesPullingTime="18.822125288s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:42.956737632 +0000 UTC m=+22.883746344" watchObservedRunningTime="2026-07-02 19:05:42.957138726 +0000 UTC m=+22.884147481" Jul 02 19:05:42.995567 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:42.995522 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hbf5k" podStartSLOduration=4.199265134 podStartE2EDuration="22.995506935s" totalImagesPullingTime="18.796241801s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:42.971101009 +0000 UTC m=+22.898109722" watchObservedRunningTime="2026-07-02 19:05:42.995506935 +0000 UTC m=+22.922515648" Jul 02 19:05:43.012037 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:43.011998 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-z8v4v" podStartSLOduration=3.162239767 podStartE2EDuration="22.01198388s" totalImagesPullingTime="18.849744113s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:43.011915023 +0000 UTC m=+22.938923735" watchObservedRunningTime="2026-07-02 19:05:43.01198388 +0000 UTC m=+22.938992591" Jul 02 19:05:43.025625 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:43.025580 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-45z57" podStartSLOduration=4.219203616 podStartE2EDuration="23.025566808s" totalImagesPullingTime="18.806363192s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:43.025466726 +0000 UTC m=+22.952475429" watchObservedRunningTime="2026-07-02 19:05:43.025566808 +0000 UTC m=+22.952575521" Jul 02 19:05:43.837796 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:43.837767 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:43.838017 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:43.837886 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9b6h" podUID="92620d26-b178-4330-a011-98af64b4d43c" Jul 02 19:05:43.939243 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:43.939207 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kpjfw" event={"ID":"bcd6dcd7-7601-48bc-a635-b32b28d95a64","Type":"ContainerStarted","Data":"389eea64fa16c3711d29a63fa2f8d39bb63a43e385a7076145e8395cb70e7fd6"} Jul 02 19:05:43.944363 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:43.944334 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" event={"ID":"85a38081-5d7f-444a-837e-f536c6049157","Type":"ContainerStarted","Data":"495933c098b5e5983470d4f477751c2ede9e854ac871e1e4b2ed2326f94b28c2"} Jul 02 19:05:43.944517 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:43.944414 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" event={"ID":"85a38081-5d7f-444a-837e-f536c6049157","Type":"ContainerStarted","Data":"7d71cf0a4b9ca40376589dfb33f93fe55fe0432dd47b0858cdf7d5f7467a9a7e"} Jul 02 19:05:43.944517 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:43.944467 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" event={"ID":"85a38081-5d7f-444a-837e-f536c6049157","Type":"ContainerStarted","Data":"2c17be64f6dea5428195760b7ae4374c279942907f247545d915041cf1f7dce8"} Jul 02 19:05:43.944517 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:43.944515 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" event={"ID":"85a38081-5d7f-444a-837e-f536c6049157","Type":"ContainerStarted","Data":"a6d3f9c1e8c9fabd4f94df8a49b784862619554135c5db6ce8f45fd8e5bb25c2"} Jul 02 19:05:43.944698 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:43.944567 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" event={"ID":"85a38081-5d7f-444a-837e-f536c6049157","Type":"ContainerStarted","Data":"1b02954f00dc4f0d377396fca1bf64ff60a040878cde4d18783c842d8594a626"} Jul 02 19:05:43.955509 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:43.955399 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kpjfw" podStartSLOduration=5.138629517 podStartE2EDuration="23.955380076s" totalImagesPullingTime="18.816750559s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:43.954997566 +0000 UTC m=+23.882006278" watchObservedRunningTime="2026-07-02 19:05:43.955380076 +0000 UTC m=+23.882388793" Jul 02 19:05:44.042892 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:44.042852 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 02 19:05:44.746668 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:44.746510 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T19:05:44.04287326Z","UUID":"544c2f28-9167-435a-9963-4b2c765e1c1a","Handler":null,"Name":"","Endpoint":""} Jul 02 19:05:44.748275 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:44.748249 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 02 19:05:44.748420 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:44.748287 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 02 19:05:44.838474 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:44.838441 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:44.838718 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:44.838691 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hssx4" podUID="4d87225d-39f9-40d0-8ae7-5fa11099c16c" Jul 02 19:05:44.839176 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:44.839150 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:44.839301 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:44.839256 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txtgt" podUID="0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef" Jul 02 19:05:44.949447 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:44.949407 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" event={"ID":"3cf388a6-8721-429f-a2f9-f5f950e40a3b","Type":"ContainerStarted","Data":"757f56fbe309031244c029c4856d1a3dea8ab9edbfe8447d6f3736cedf767181"} Jul 02 19:05:45.683236 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:45.683161 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bl8kt" Jul 02 19:05:45.683855 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:45.683833 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bl8kt" Jul 02 19:05:45.838094 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:45.838058 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:45.838284 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:45.838259 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9b6h" podUID="92620d26-b178-4330-a011-98af64b4d43c" Jul 02 19:05:45.954752 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:45.954657 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" event={"ID":"3cf388a6-8721-429f-a2f9-f5f950e40a3b","Type":"ContainerStarted","Data":"9cf4a311321a2e34b39b8d78533999a7e626b4325e4793b6b76505a6b637e60f"} Jul 02 19:05:45.958710 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:45.958664 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" event={"ID":"85a38081-5d7f-444a-837e-f536c6049157","Type":"ContainerStarted","Data":"9388aee18a1aff8a3c6889f9eeef3fe01d329ea18c91b06d94d5454fe391424d"} Jul 02 19:05:45.958888 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:45.958863 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bl8kt" Jul 02 19:05:45.959355 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:45.959337 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bl8kt" Jul 02 19:05:45.989894 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:45.989836 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mbwg" podStartSLOduration=4.995014868 podStartE2EDuration="25.989817643s" totalImagesPullingTime="20.994802775s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:45.989596355 +0000 UTC m=+25.916605066" watchObservedRunningTime="2026-07-02 19:05:45.989817643 +0000 UTC m=+25.916826358" Jul 02 19:05:46.838351 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:46.838140 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:46.838567 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:46.838184 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:46.838567 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:46.838459 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txtgt" podUID="0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef" Jul 02 19:05:46.838567 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:46.838505 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hssx4" podUID="4d87225d-39f9-40d0-8ae7-5fa11099c16c" Jul 02 19:05:47.838694 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:47.838435 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:47.839264 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:47.838721 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9b6h" podUID="92620d26-b178-4330-a011-98af64b4d43c" Jul 02 19:05:47.965809 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:47.965775 2566 generic.go:356] "Generic (PLEG): container finished" podID="8e4ef969-3b7b-41a4-865b-f15c27d2cd69" containerID="39366876858ba17c5fc6198215b199855ea42826a4e6f12c532e89257bb73481" exitCode=0 Jul 02 19:05:47.966047 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:47.965861 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6nnrq" event={"ID":"8e4ef969-3b7b-41a4-865b-f15c27d2cd69","Type":"ContainerDied","Data":"39366876858ba17c5fc6198215b199855ea42826a4e6f12c532e89257bb73481"} Jul 02 19:05:47.969487 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:47.969457 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" event={"ID":"85a38081-5d7f-444a-837e-f536c6049157","Type":"ContainerStarted","Data":"f14096cfe7324d2ca1d06026266d4fe2b585d42df12b26f45555aed10c98f87f"} Jul 02 19:05:47.969718 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:47.969704 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:47.969794 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:47.969754 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:47.985686 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:47.985654 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:48.015507 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:48.015464 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" podStartSLOduration=9.014217129 podStartE2EDuration="28.015449727s" totalImagesPullingTime="19.001232598s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:48.015419687 +0000 UTC m=+27.942428400" watchObservedRunningTime="2026-07-02 19:05:48.015449727 +0000 UTC m=+27.942458420" Jul 02 19:05:48.838693 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:48.838649 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:48.839163 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:48.838788 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txtgt" podUID="0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef" Jul 02 19:05:48.839163 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:48.838653 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:48.839272 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:48.839225 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hssx4" podUID="4d87225d-39f9-40d0-8ae7-5fa11099c16c" Jul 02 19:05:48.972295 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:48.972259 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:48.987419 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:48.987393 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:05:49.548829 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:49.548542 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hssx4"] Jul 02 19:05:49.549008 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:49.548864 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:49.549008 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:49.548961 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hssx4" podUID="4d87225d-39f9-40d0-8ae7-5fa11099c16c" Jul 02 19:05:49.552055 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:49.552029 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c9b6h"] Jul 02 19:05:49.552205 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:49.552157 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:49.552269 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:49.552250 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9b6h" podUID="92620d26-b178-4330-a011-98af64b4d43c" Jul 02 19:05:49.552721 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:49.552693 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-txtgt"] Jul 02 19:05:49.552829 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:49.552784 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:49.552898 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:49.552873 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txtgt" podUID="0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef" Jul 02 19:05:49.976152 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:49.976069 2566 generic.go:356] "Generic (PLEG): container finished" podID="8e4ef969-3b7b-41a4-865b-f15c27d2cd69" containerID="2bb097086c8cc034cb9778ffcf23057e79e32947c742172f497436fb4cdce407" exitCode=0 Jul 02 19:05:49.976554 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:49.976157 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6nnrq" event={"ID":"8e4ef969-3b7b-41a4-865b-f15c27d2cd69","Type":"ContainerDied","Data":"2bb097086c8cc034cb9778ffcf23057e79e32947c742172f497436fb4cdce407"} Jul 02 19:05:50.838687 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:50.838652 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:50.838896 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:50.838763 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txtgt" podUID="0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef" Jul 02 19:05:50.838896 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:50.838787 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:50.838896 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:50.838819 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:50.839012 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:50.838890 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9b6h" podUID="92620d26-b178-4330-a011-98af64b4d43c" Jul 02 19:05:50.839012 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:50.838965 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hssx4" podUID="4d87225d-39f9-40d0-8ae7-5fa11099c16c" Jul 02 19:05:51.983891 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:51.983864 2566 generic.go:356] "Generic (PLEG): container finished" podID="8e4ef969-3b7b-41a4-865b-f15c27d2cd69" containerID="af6b576a4ff2519ccb0483d76447e92a1421d2892f3e5c0b71c89dd4d303d3ab" exitCode=0 Jul 02 19:05:51.984444 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:51.983911 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6nnrq" event={"ID":"8e4ef969-3b7b-41a4-865b-f15c27d2cd69","Type":"ContainerDied","Data":"af6b576a4ff2519ccb0483d76447e92a1421d2892f3e5c0b71c89dd4d303d3ab"} Jul 02 19:05:52.838361 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:52.838321 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:52.838547 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:52.838376 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:52.838547 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:52.838321 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:52.838547 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:52.838471 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9b6h" podUID="92620d26-b178-4330-a011-98af64b4d43c" Jul 02 19:05:52.838547 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:52.838530 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txtgt" podUID="0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef" Jul 02 19:05:52.838769 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:52.838628 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hssx4" podUID="4d87225d-39f9-40d0-8ae7-5fa11099c16c" Jul 02 19:05:54.438976 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:54.438942 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs\") pod \"network-metrics-daemon-c9b6h\" (UID: \"92620d26-b178-4330-a011-98af64b4d43c\") " pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:54.439446 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:54.439088 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:05:54.439446 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:54.439173 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs podName:92620d26-b178-4330-a011-98af64b4d43c nodeName:}" failed. No retries permitted until 2026-07-02 19:06:26.439153204 +0000 UTC m=+66.366161893 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs") pod "network-metrics-daemon-c9b6h" (UID: "92620d26-b178-4330-a011-98af64b4d43c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 19:05:54.539505 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:54.539469 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret\") pod \"global-pull-secret-syncer-txtgt\" (UID: \"0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef\") " pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:54.539701 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:54.539521 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhqhb\" (UniqueName: \"kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb\") pod \"network-check-target-hssx4\" (UID: \"4d87225d-39f9-40d0-8ae7-5fa11099c16c\") " pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:54.539701 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:54.539640 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 19:05:54.539701 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:54.539660 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 19:05:54.539701 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:54.539670 2566 projected.go:196] Error preparing data for projected volume kube-api-access-xhqhb for pod openshift-network-diagnostics/network-check-target-hssx4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:05:54.539858 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:54.539640 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 19:05:54.539858 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:54.539736 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb podName:4d87225d-39f9-40d0-8ae7-5fa11099c16c nodeName:}" failed. No retries permitted until 2026-07-02 19:06:26.539718292 +0000 UTC m=+66.466726986 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhqhb" (UniqueName: "kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb") pod "network-check-target-hssx4" (UID: "4d87225d-39f9-40d0-8ae7-5fa11099c16c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 19:05:54.539858 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:54.539781 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret podName:0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef nodeName:}" failed. No retries permitted until 2026-07-02 19:06:26.53976345 +0000 UTC m=+66.466772154 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret") pod "global-pull-secret-syncer-txtgt" (UID: "0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef") : object "kube-system"/"original-pull-secret" not registered Jul 02 19:05:54.838592 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:54.838557 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:54.838792 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:54.838557 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:54.838792 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:54.838719 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c9b6h" podUID="92620d26-b178-4330-a011-98af64b4d43c" Jul 02 19:05:54.838792 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:54.838574 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:54.838792 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:54.838781 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txtgt" podUID="0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef" Jul 02 19:05:54.838987 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:54.838852 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hssx4" podUID="4d87225d-39f9-40d0-8ae7-5fa11099c16c" Jul 02 19:05:55.309739 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.309706 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-205.ec2.internal" event="NodeReady" Jul 02 19:05:55.309895 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.309868 2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 02 19:05:55.355098 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.355056 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-s5ft7"] Jul 02 19:05:55.357507 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.357474 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-66wg5"] Jul 02 19:05:55.357616 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.357602 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s5ft7" Jul 02 19:05:55.360048 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.360024 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 02 19:05:55.360169 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.360078 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-h97jw\"" Jul 02 19:05:55.360825 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.360596 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 02 19:05:55.360825 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.360742 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 02 19:05:55.360825 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.360593 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-66wg5" Jul 02 19:05:55.363051 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.363010 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-crw8l\"" Jul 02 19:05:55.363051 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.363030 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 02 19:05:55.363208 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.363125 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 02 19:05:55.366436 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.366403 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s5ft7"] Jul 02 19:05:55.369379 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.369323 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-66wg5"] Jul 02 19:05:55.446310 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.446270 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5tfq\" (UniqueName: \"kubernetes.io/projected/7ac120d5-4418-44db-adea-9c186a1a46af-kube-api-access-b5tfq\") pod \"ingress-canary-s5ft7\" (UID: \"7ac120d5-4418-44db-adea-9c186a1a46af\") " pod="openshift-ingress-canary/ingress-canary-s5ft7" Jul 02 19:05:55.446750 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.446344 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc7p8\" (UniqueName: \"kubernetes.io/projected/3c79663f-9a62-40e4-b870-f2f156b654d0-kube-api-access-lc7p8\") pod \"dns-default-66wg5\" (UID: \"3c79663f-9a62-40e4-b870-f2f156b654d0\") " pod="openshift-dns/dns-default-66wg5" Jul 02 19:05:55.446750 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.446394 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3c79663f-9a62-40e4-b870-f2f156b654d0-config-volume\") pod \"dns-default-66wg5\" (UID: \"3c79663f-9a62-40e4-b870-f2f156b654d0\") " pod="openshift-dns/dns-default-66wg5" Jul 02 19:05:55.446750 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.446491 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c79663f-9a62-40e4-b870-f2f156b654d0-metrics-tls\") pod \"dns-default-66wg5\" (UID: \"3c79663f-9a62-40e4-b870-f2f156b654d0\") " pod="openshift-dns/dns-default-66wg5" Jul 02 19:05:55.446750 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.446517 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c79663f-9a62-40e4-b870-f2f156b654d0-tmp-dir\") pod \"dns-default-66wg5\" (UID: \"3c79663f-9a62-40e4-b870-f2f156b654d0\") " pod="openshift-dns/dns-default-66wg5" Jul 02 19:05:55.446750 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.446539 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ac120d5-4418-44db-adea-9c186a1a46af-cert\") pod \"ingress-canary-s5ft7\" (UID: \"7ac120d5-4418-44db-adea-9c186a1a46af\") " pod="openshift-ingress-canary/ingress-canary-s5ft7" Jul 02 19:05:55.547600 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.547561 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c79663f-9a62-40e4-b870-f2f156b654d0-metrics-tls\") pod \"dns-default-66wg5\" (UID: \"3c79663f-9a62-40e4-b870-f2f156b654d0\") " pod="openshift-dns/dns-default-66wg5" Jul 02 19:05:55.547600 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.547604 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c79663f-9a62-40e4-b870-f2f156b654d0-tmp-dir\") pod \"dns-default-66wg5\" (UID: \"3c79663f-9a62-40e4-b870-f2f156b654d0\") " pod="openshift-dns/dns-default-66wg5" Jul 02 19:05:55.547832 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.547635 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ac120d5-4418-44db-adea-9c186a1a46af-cert\") pod \"ingress-canary-s5ft7\" (UID: \"7ac120d5-4418-44db-adea-9c186a1a46af\") " pod="openshift-ingress-canary/ingress-canary-s5ft7" Jul 02 19:05:55.547832 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.547696 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5tfq\" (UniqueName: \"kubernetes.io/projected/7ac120d5-4418-44db-adea-9c186a1a46af-kube-api-access-b5tfq\") pod \"ingress-canary-s5ft7\" (UID: \"7ac120d5-4418-44db-adea-9c186a1a46af\") " pod="openshift-ingress-canary/ingress-canary-s5ft7" Jul 02 19:05:55.547832 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.547749 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc7p8\" (UniqueName: \"kubernetes.io/projected/3c79663f-9a62-40e4-b870-f2f156b654d0-kube-api-access-lc7p8\") pod \"dns-default-66wg5\" (UID: \"3c79663f-9a62-40e4-b870-f2f156b654d0\") " pod="openshift-dns/dns-default-66wg5" Jul 02 19:05:55.547832 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:55.547756 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 19:05:55.547832 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.547797 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3c79663f-9a62-40e4-b870-f2f156b654d0-config-volume\") pod \"dns-default-66wg5\" (UID: \"3c79663f-9a62-40e4-b870-f2f156b654d0\") " pod="openshift-dns/dns-default-66wg5" Jul 02 19:05:55.548050 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:55.547836 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c79663f-9a62-40e4-b870-f2f156b654d0-metrics-tls podName:3c79663f-9a62-40e4-b870-f2f156b654d0 nodeName:}" failed. No retries permitted until 2026-07-02 19:05:56.047811961 +0000 UTC m=+35.974820664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c79663f-9a62-40e4-b870-f2f156b654d0-metrics-tls") pod "dns-default-66wg5" (UID: "3c79663f-9a62-40e4-b870-f2f156b654d0") : secret "dns-default-metrics-tls" not found Jul 02 19:05:55.548050 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.547965 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c79663f-9a62-40e4-b870-f2f156b654d0-tmp-dir\") pod \"dns-default-66wg5\" (UID: \"3c79663f-9a62-40e4-b870-f2f156b654d0\") " pod="openshift-dns/dns-default-66wg5" Jul 02 19:05:55.548537 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.548294 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3c79663f-9a62-40e4-b870-f2f156b654d0-config-volume\") pod \"dns-default-66wg5\" (UID: \"3c79663f-9a62-40e4-b870-f2f156b654d0\") " pod="openshift-dns/dns-default-66wg5" Jul 02 19:05:55.553038 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.553009 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ac120d5-4418-44db-adea-9c186a1a46af-cert\") pod \"ingress-canary-s5ft7\" (UID: \"7ac120d5-4418-44db-adea-9c186a1a46af\") " pod="openshift-ingress-canary/ingress-canary-s5ft7" Jul 02 19:05:55.556377 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.556356 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc7p8\" (UniqueName: \"kubernetes.io/projected/3c79663f-9a62-40e4-b870-f2f156b654d0-kube-api-access-lc7p8\") pod \"dns-default-66wg5\" (UID: \"3c79663f-9a62-40e4-b870-f2f156b654d0\") " pod="openshift-dns/dns-default-66wg5" Jul 02 19:05:55.556497 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.556410 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5tfq\" (UniqueName: \"kubernetes.io/projected/7ac120d5-4418-44db-adea-9c186a1a46af-kube-api-access-b5tfq\") pod \"ingress-canary-s5ft7\" (UID: \"7ac120d5-4418-44db-adea-9c186a1a46af\") " pod="openshift-ingress-canary/ingress-canary-s5ft7" Jul 02 19:05:55.671224 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:55.671120 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s5ft7" Jul 02 19:05:56.021212 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:56.021187 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s5ft7"] Jul 02 19:05:56.025852 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:05:56.025825 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ac120d5_4418_44db_adea_9c186a1a46af.slice/crio-fb163a929f1a15f7c361f2d799aace0494d28e6348e7937f98bf4ead7889c37a WatchSource:0}: Error finding container fb163a929f1a15f7c361f2d799aace0494d28e6348e7937f98bf4ead7889c37a: Status 404 returned error can't find the container with id fb163a929f1a15f7c361f2d799aace0494d28e6348e7937f98bf4ead7889c37a Jul 02 19:05:56.051558 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:56.051523 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c79663f-9a62-40e4-b870-f2f156b654d0-metrics-tls\") pod \"dns-default-66wg5\" (UID: \"3c79663f-9a62-40e4-b870-f2f156b654d0\") " pod="openshift-dns/dns-default-66wg5" Jul 02 19:05:56.051756 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:56.051633 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 19:05:56.051756 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:56.051704 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c79663f-9a62-40e4-b870-f2f156b654d0-metrics-tls podName:3c79663f-9a62-40e4-b870-f2f156b654d0 nodeName:}" failed. No retries permitted until 2026-07-02 19:05:57.051685432 +0000 UTC m=+36.978694137 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c79663f-9a62-40e4-b870-f2f156b654d0-metrics-tls") pod "dns-default-66wg5" (UID: "3c79663f-9a62-40e4-b870-f2f156b654d0") : secret "dns-default-metrics-tls" not found Jul 02 19:05:56.838878 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:56.838796 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:05:56.839299 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:56.838796 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:05:56.839299 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:56.838802 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:05:56.842917 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:56.842887 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 19:05:56.842917 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:56.842887 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 19:05:56.843121 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:56.842937 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mxlv5\"" Jul 02 19:05:56.843121 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:56.842962 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 19:05:56.843121 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:56.842897 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 19:05:56.843121 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:56.842889 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jjgvz\"" Jul 02 19:05:56.997752 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:56.997698 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s5ft7" event={"ID":"7ac120d5-4418-44db-adea-9c186a1a46af","Type":"ContainerStarted","Data":"fb163a929f1a15f7c361f2d799aace0494d28e6348e7937f98bf4ead7889c37a"} Jul 02 19:05:57.000770 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:57.000744 2566 generic.go:356] "Generic (PLEG): container finished" podID="8e4ef969-3b7b-41a4-865b-f15c27d2cd69" containerID="bdc4abdbf75abe8c0ca592f42b4a1b4fc342e9a870c1b3bf0aec739b0c05e5c2" exitCode=0 Jul 02 19:05:57.000911 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:57.000777 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6nnrq" event={"ID":"8e4ef969-3b7b-41a4-865b-f15c27d2cd69","Type":"ContainerDied","Data":"bdc4abdbf75abe8c0ca592f42b4a1b4fc342e9a870c1b3bf0aec739b0c05e5c2"} Jul 02 19:05:57.058291 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:57.057867 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c79663f-9a62-40e4-b870-f2f156b654d0-metrics-tls\") pod \"dns-default-66wg5\" (UID: \"3c79663f-9a62-40e4-b870-f2f156b654d0\") " pod="openshift-dns/dns-default-66wg5" Jul 02 19:05:57.058291 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:57.058055 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 19:05:57.058291 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:57.058122 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c79663f-9a62-40e4-b870-f2f156b654d0-metrics-tls podName:3c79663f-9a62-40e4-b870-f2f156b654d0 nodeName:}" failed. No retries permitted until 2026-07-02 19:05:59.058102636 +0000 UTC m=+38.985111326 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c79663f-9a62-40e4-b870-f2f156b654d0-metrics-tls") pod "dns-default-66wg5" (UID: "3c79663f-9a62-40e4-b870-f2f156b654d0") : secret "dns-default-metrics-tls" not found Jul 02 19:05:58.006534 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:58.006326 2566 generic.go:356] "Generic (PLEG): container finished" podID="8e4ef969-3b7b-41a4-865b-f15c27d2cd69" containerID="060268f0b243d06a6a2a6d9e424827016d85a54ed1f109716553741f96f527ad" exitCode=0 Jul 02 19:05:58.006534 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:58.006402 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6nnrq" event={"ID":"8e4ef969-3b7b-41a4-865b-f15c27d2cd69","Type":"ContainerDied","Data":"060268f0b243d06a6a2a6d9e424827016d85a54ed1f109716553741f96f527ad"} Jul 02 19:05:58.146625 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:58.146590 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:05:58.229414 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:58.229383 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:05:58.420163 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:58.420120 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:05:58.641521 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:58.641489 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:05:58.739575 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:58.739544 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:05:59.010612 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:59.010580 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s5ft7" event={"ID":"7ac120d5-4418-44db-adea-9c186a1a46af","Type":"ContainerStarted","Data":"3a6c54fb703661a990f41824e49a2b2198850e24b5a1f74c33ee4e25126fae39"} Jul 02 19:05:59.013642 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:59.013620 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6nnrq" event={"ID":"8e4ef969-3b7b-41a4-865b-f15c27d2cd69","Type":"ContainerStarted","Data":"b0f7bcdcc79e41b6fa045db1b10f14effce19a4aca73ab42ac039059f632d5cc"} Jul 02 19:05:59.029427 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:59.029378 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-s5ft7" podStartSLOduration=1.546633131 podStartE2EDuration="4.029361502s" totalImagesPullingTime="2.482728371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:05:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:59.029119446 +0000 UTC m=+38.956128182" watchObservedRunningTime="2026-07-02 19:05:59.029361502 +0000 UTC m=+38.956370215" Jul 02 19:05:59.050133 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:59.050070 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6nnrq" podStartSLOduration=9.563779893 podStartE2EDuration="39.050057972s" totalImagesPullingTime="29.486278079s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:20 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:05:59.049020529 +0000 UTC m=+38.976029241" watchObservedRunningTime="2026-07-02 19:05:59.050057972 +0000 UTC m=+38.977066684" Jul 02 19:05:59.074804 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:59.074777 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c79663f-9a62-40e4-b870-f2f156b654d0-metrics-tls\") pod \"dns-default-66wg5\" (UID: \"3c79663f-9a62-40e4-b870-f2f156b654d0\") " pod="openshift-dns/dns-default-66wg5" Jul 02 19:05:59.074936 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:59.074883 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 19:05:59.074936 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:05:59.074931 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c79663f-9a62-40e4-b870-f2f156b654d0-metrics-tls podName:3c79663f-9a62-40e4-b870-f2f156b654d0 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:03.074917278 +0000 UTC m=+43.001925968 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c79663f-9a62-40e4-b870-f2f156b654d0-metrics-tls") pod "dns-default-66wg5" (UID: "3c79663f-9a62-40e4-b870-f2f156b654d0") : secret "dns-default-metrics-tls" not found Jul 02 19:05:59.169967 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:59.169926 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:05:59.230360 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:59.230332 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:05:59.409328 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:59.409233 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:05:59.472133 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:05:59.472094 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:00.933783 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:00.933751 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:01.000526 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:01.000470 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:03.101869 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:03.101832 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c79663f-9a62-40e4-b870-f2f156b654d0-metrics-tls\") pod \"dns-default-66wg5\" (UID: \"3c79663f-9a62-40e4-b870-f2f156b654d0\") " pod="openshift-dns/dns-default-66wg5" Jul 02 19:06:03.102322 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:03.102021 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 19:06:03.102322 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:03.102101 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c79663f-9a62-40e4-b870-f2f156b654d0-metrics-tls podName:3c79663f-9a62-40e4-b870-f2f156b654d0 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:11.102079332 +0000 UTC m=+51.029088040 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c79663f-9a62-40e4-b870-f2f156b654d0-metrics-tls") pod "dns-default-66wg5" (UID: "3c79663f-9a62-40e4-b870-f2f156b654d0") : secret "dns-default-metrics-tls" not found Jul 02 19:06:04.139978 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.139946 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:04.196413 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.196388 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:04.369352 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.369318 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:04.428499 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.428422 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-d29gg"] Jul 02 19:06:04.459798 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.459771 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:04.460827 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.460812 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d29gg"] Jul 02 19:06:04.460930 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.460918 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:04.463797 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.463775 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 02 19:06:04.464191 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.464178 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 02 19:06:04.464446 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.464430 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 02 19:06:04.465193 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.465119 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 02 19:06:04.465407 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.465383 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-sbf27\"" Jul 02 19:06:04.565172 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.565134 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:04.609246 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.609212 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d13f0e4-7378-429b-a212-431f984561db-data-volume\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:04.609372 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.609250 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6d13f0e4-7378-429b-a212-431f984561db-crio-socket\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:04.609372 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.609272 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t769\" (UniqueName: \"kubernetes.io/projected/6d13f0e4-7378-429b-a212-431f984561db-kube-api-access-8t769\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:04.609372 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.609328 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d13f0e4-7378-429b-a212-431f984561db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:04.609486 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.609373 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6d13f0e4-7378-429b-a212-431f984561db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:04.665108 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.665078 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:04.709914 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.709844 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6d13f0e4-7378-429b-a212-431f984561db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:04.710046 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.709927 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d13f0e4-7378-429b-a212-431f984561db-data-volume\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:04.710046 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.709949 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6d13f0e4-7378-429b-a212-431f984561db-crio-socket\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:04.710046 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:04.709969 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 19:06:04.710046 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.709975 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t769\" (UniqueName: \"kubernetes.io/projected/6d13f0e4-7378-429b-a212-431f984561db-kube-api-access-8t769\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:04.710046 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.710010 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d13f0e4-7378-429b-a212-431f984561db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:04.710283 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:04.710068 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6d13f0e4-7378-429b-a212-431f984561db-kube-rbac-proxy-cm podName:6d13f0e4-7378-429b-a212-431f984561db nodeName:}" failed. No retries permitted until 2026-07-02 19:06:05.210053233 +0000 UTC m=+45.137061923 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6d13f0e4-7378-429b-a212-431f984561db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d29gg" (UID: "6d13f0e4-7378-429b-a212-431f984561db") : configmap "kube-rbac-proxy" not found Jul 02 19:06:04.710283 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:04.710134 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 19:06:04.710283 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.710161 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6d13f0e4-7378-429b-a212-431f984561db-crio-socket\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:04.710283 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:04.710206 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d13f0e4-7378-429b-a212-431f984561db-insights-runtime-extractor-tls podName:6d13f0e4-7378-429b-a212-431f984561db nodeName:}" failed. No retries permitted until 2026-07-02 19:06:05.210187667 +0000 UTC m=+45.137196368 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6d13f0e4-7378-429b-a212-431f984561db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d29gg" (UID: "6d13f0e4-7378-429b-a212-431f984561db") : secret "insights-runtime-extractor-tls" not found Jul 02 19:06:04.710430 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.710357 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d13f0e4-7378-429b-a212-431f984561db-data-volume\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:04.719552 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:04.719530 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t769\" (UniqueName: \"kubernetes.io/projected/6d13f0e4-7378-429b-a212-431f984561db-kube-api-access-8t769\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:05.048451 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:05.048416 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:05.142280 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:05.142242 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:05.214447 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:05.214413 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d13f0e4-7378-429b-a212-431f984561db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:05.214587 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:05.214458 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6d13f0e4-7378-429b-a212-431f984561db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:05.214587 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:05.214567 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 19:06:05.214587 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:05.214576 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 19:06:05.214727 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:05.214638 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6d13f0e4-7378-429b-a212-431f984561db-kube-rbac-proxy-cm podName:6d13f0e4-7378-429b-a212-431f984561db nodeName:}" failed. No retries permitted until 2026-07-02 19:06:06.214620568 +0000 UTC m=+46.141629259 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6d13f0e4-7378-429b-a212-431f984561db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d29gg" (UID: "6d13f0e4-7378-429b-a212-431f984561db") : configmap "kube-rbac-proxy" not found Jul 02 19:06:05.214727 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:05.214653 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d13f0e4-7378-429b-a212-431f984561db-insights-runtime-extractor-tls podName:6d13f0e4-7378-429b-a212-431f984561db nodeName:}" failed. No retries permitted until 2026-07-02 19:06:06.214645741 +0000 UTC m=+46.141654431 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6d13f0e4-7378-429b-a212-431f984561db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d29gg" (UID: "6d13f0e4-7378-429b-a212-431f984561db") : secret "insights-runtime-extractor-tls" not found Jul 02 19:06:05.489864 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:05.489834 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-2kswt"] Jul 02 19:06:05.505549 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:05.505519 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-2kswt"] Jul 02 19:06:05.505740 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:05.505649 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2kswt" Jul 02 19:06:05.508204 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:05.508182 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 02 19:06:05.508306 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:05.508186 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 02 19:06:05.509060 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:05.509037 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ppmvq\"" Jul 02 19:06:05.617538 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:05.617504 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxphk\" (UniqueName: \"kubernetes.io/projected/b544ec3c-7ef0-4490-8c4f-87384346634f-kube-api-access-jxphk\") pod \"migrator-d57d58cc9-2kswt\" (UID: \"b544ec3c-7ef0-4490-8c4f-87384346634f\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2kswt" Jul 02 19:06:05.702310 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:05.702241 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:05.718704 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:05.718658 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxphk\" (UniqueName: \"kubernetes.io/projected/b544ec3c-7ef0-4490-8c4f-87384346634f-kube-api-access-jxphk\") pod \"migrator-d57d58cc9-2kswt\" (UID: \"b544ec3c-7ef0-4490-8c4f-87384346634f\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2kswt" Jul 02 19:06:05.726964 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:05.726937 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxphk\" (UniqueName: \"kubernetes.io/projected/b544ec3c-7ef0-4490-8c4f-87384346634f-kube-api-access-jxphk\") pod \"migrator-d57d58cc9-2kswt\" (UID: \"b544ec3c-7ef0-4490-8c4f-87384346634f\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2kswt" Jul 02 19:06:05.769217 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:05.769138 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:05.814583 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:05.814556 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2kswt" Jul 02 19:06:05.957367 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:05.957337 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-2kswt"] Jul 02 19:06:05.960955 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:06:05.960924 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb544ec3c_7ef0_4490_8c4f_87384346634f.slice/crio-727d947347d77231aaacebe6c84cd7cd88257878fc85da93a8792c78e78deb54 WatchSource:0}: Error finding container 727d947347d77231aaacebe6c84cd7cd88257878fc85da93a8792c78e78deb54: Status 404 returned error can't find the container with id 727d947347d77231aaacebe6c84cd7cd88257878fc85da93a8792c78e78deb54 Jul 02 19:06:06.030858 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.030786 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2kswt" event={"ID":"b544ec3c-7ef0-4490-8c4f-87384346634f","Type":"ContainerStarted","Data":"727d947347d77231aaacebe6c84cd7cd88257878fc85da93a8792c78e78deb54"} Jul 02 19:06:06.223168 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.223131 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d13f0e4-7378-429b-a212-431f984561db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:06.223168 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.223174 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6d13f0e4-7378-429b-a212-431f984561db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:06.223568 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:06.223238 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 19:06:06.223568 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:06.223278 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 19:06:06.223568 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:06.223286 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6d13f0e4-7378-429b-a212-431f984561db-kube-rbac-proxy-cm podName:6d13f0e4-7378-429b-a212-431f984561db nodeName:}" failed. No retries permitted until 2026-07-02 19:06:08.223271493 +0000 UTC m=+48.150280183 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6d13f0e4-7378-429b-a212-431f984561db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d29gg" (UID: "6d13f0e4-7378-429b-a212-431f984561db") : configmap "kube-rbac-proxy" not found Jul 02 19:06:06.223568 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:06.223392 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d13f0e4-7378-429b-a212-431f984561db-insights-runtime-extractor-tls podName:6d13f0e4-7378-429b-a212-431f984561db nodeName:}" failed. No retries permitted until 2026-07-02 19:06:08.223379228 +0000 UTC m=+48.150387919 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6d13f0e4-7378-429b-a212-431f984561db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d29gg" (UID: "6d13f0e4-7378-429b-a212-431f984561db") : secret "insights-runtime-extractor-tls" not found Jul 02 19:06:06.403696 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.403609 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hbf5k_a04cbb97-93a7-4d61-bdf9-deba1968e7c0/dns-node-resolver/0.log" Jul 02 19:06:06.661120 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.661044 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-82vw5"] Jul 02 19:06:06.683873 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.683849 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-82vw5"] Jul 02 19:06:06.683998 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.683934 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-82vw5" Jul 02 19:06:06.686357 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.686327 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 02 19:06:06.686357 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.686349 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-7m69j\"" Jul 02 19:06:06.687318 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.687294 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 02 19:06:06.687423 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.687333 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 02 19:06:06.687423 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.687350 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 02 19:06:06.693126 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.693108 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:06.726983 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.726958 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/975ecd23-87b4-4897-b7c4-deeaa4487331-signing-key\") pod \"service-ca-758f599c5-82vw5\" (UID: \"975ecd23-87b4-4897-b7c4-deeaa4487331\") " pod="openshift-service-ca/service-ca-758f599c5-82vw5" Jul 02 19:06:06.727075 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.726990 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd2qd\" (UniqueName: \"kubernetes.io/projected/975ecd23-87b4-4897-b7c4-deeaa4487331-kube-api-access-vd2qd\") pod \"service-ca-758f599c5-82vw5\" (UID: \"975ecd23-87b4-4897-b7c4-deeaa4487331\") " pod="openshift-service-ca/service-ca-758f599c5-82vw5" Jul 02 19:06:06.727075 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.727029 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/975ecd23-87b4-4897-b7c4-deeaa4487331-signing-cabundle\") pod \"service-ca-758f599c5-82vw5\" (UID: \"975ecd23-87b4-4897-b7c4-deeaa4487331\") " pod="openshift-service-ca/service-ca-758f599c5-82vw5" Jul 02 19:06:06.794948 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.794917 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:06.827470 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.827436 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/975ecd23-87b4-4897-b7c4-deeaa4487331-signing-cabundle\") pod \"service-ca-758f599c5-82vw5\" (UID: \"975ecd23-87b4-4897-b7c4-deeaa4487331\") " pod="openshift-service-ca/service-ca-758f599c5-82vw5" Jul 02 19:06:06.827660 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.827642 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/975ecd23-87b4-4897-b7c4-deeaa4487331-signing-key\") pod \"service-ca-758f599c5-82vw5\" (UID: \"975ecd23-87b4-4897-b7c4-deeaa4487331\") " pod="openshift-service-ca/service-ca-758f599c5-82vw5" Jul 02 19:06:06.827748 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.827695 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd2qd\" (UniqueName: \"kubernetes.io/projected/975ecd23-87b4-4897-b7c4-deeaa4487331-kube-api-access-vd2qd\") pod \"service-ca-758f599c5-82vw5\" (UID: \"975ecd23-87b4-4897-b7c4-deeaa4487331\") " pod="openshift-service-ca/service-ca-758f599c5-82vw5" Jul 02 19:06:06.828133 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.828112 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/975ecd23-87b4-4897-b7c4-deeaa4487331-signing-cabundle\") pod \"service-ca-758f599c5-82vw5\" (UID: \"975ecd23-87b4-4897-b7c4-deeaa4487331\") " pod="openshift-service-ca/service-ca-758f599c5-82vw5" Jul 02 19:06:06.830028 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.830010 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/975ecd23-87b4-4897-b7c4-deeaa4487331-signing-key\") pod \"service-ca-758f599c5-82vw5\" (UID: \"975ecd23-87b4-4897-b7c4-deeaa4487331\") " pod="openshift-service-ca/service-ca-758f599c5-82vw5" Jul 02 19:06:06.835884 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.835862 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd2qd\" (UniqueName: \"kubernetes.io/projected/975ecd23-87b4-4897-b7c4-deeaa4487331-kube-api-access-vd2qd\") pod \"service-ca-758f599c5-82vw5\" (UID: \"975ecd23-87b4-4897-b7c4-deeaa4487331\") " pod="openshift-service-ca/service-ca-758f599c5-82vw5" Jul 02 19:06:06.993051 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:06.993012 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-82vw5" Jul 02 19:06:07.403633 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:07.403603 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-45z57_d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e/node-ca/0.log" Jul 02 19:06:07.447068 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:07.447023 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-82vw5"] Jul 02 19:06:07.449516 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:06:07.449486 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod975ecd23_87b4_4897_b7c4_deeaa4487331.slice/crio-880cfdb24d459a90ec40ae4aa75021487ac2a26bf4a3382f4d1842fc80d768bd WatchSource:0}: Error finding container 880cfdb24d459a90ec40ae4aa75021487ac2a26bf4a3382f4d1842fc80d768bd: Status 404 returned error can't find the container with id 880cfdb24d459a90ec40ae4aa75021487ac2a26bf4a3382f4d1842fc80d768bd Jul 02 19:06:07.834573 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:07.834545 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:08.038806 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:08.038758 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2kswt" event={"ID":"b544ec3c-7ef0-4490-8c4f-87384346634f","Type":"ContainerStarted","Data":"3f658c3678bee974b7fada951c361e0a893398e501f0a657deebc8a3cfa38ff5"} Jul 02 19:06:08.038995 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:08.038834 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2kswt" event={"ID":"b544ec3c-7ef0-4490-8c4f-87384346634f","Type":"ContainerStarted","Data":"092c90acf6d0aca9548f6d85c79dc62a368f44c80c6a95447b1df5beb6fef4b2"} Jul 02 19:06:08.040042 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:08.040010 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-82vw5" event={"ID":"975ecd23-87b4-4897-b7c4-deeaa4487331","Type":"ContainerStarted","Data":"880cfdb24d459a90ec40ae4aa75021487ac2a26bf4a3382f4d1842fc80d768bd"} Jul 02 19:06:08.058811 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:08.058763 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2kswt" podStartSLOduration=1.357429615 podStartE2EDuration="3.058748891s" totalImagesPullingTime="1.701319276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:08.057860644 +0000 UTC m=+47.984869357" watchObservedRunningTime="2026-07-02 19:06:08.058748891 +0000 UTC m=+47.985757603" Jul 02 19:06:08.236717 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:08.236657 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d13f0e4-7378-429b-a212-431f984561db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:08.236906 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:08.236755 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6d13f0e4-7378-429b-a212-431f984561db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:08.236906 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:08.236818 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 19:06:08.236906 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:08.236837 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 19:06:08.236906 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:08.236894 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d13f0e4-7378-429b-a212-431f984561db-insights-runtime-extractor-tls podName:6d13f0e4-7378-429b-a212-431f984561db nodeName:}" failed. No retries permitted until 2026-07-02 19:06:12.236875053 +0000 UTC m=+52.163883771 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6d13f0e4-7378-429b-a212-431f984561db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d29gg" (UID: "6d13f0e4-7378-429b-a212-431f984561db") : secret "insights-runtime-extractor-tls" not found Jul 02 19:06:08.237112 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:08.236954 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6d13f0e4-7378-429b-a212-431f984561db-kube-rbac-proxy-cm podName:6d13f0e4-7378-429b-a212-431f984561db nodeName:}" failed. No retries permitted until 2026-07-02 19:06:12.236937513 +0000 UTC m=+52.163946215 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6d13f0e4-7378-429b-a212-431f984561db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-d29gg" (UID: "6d13f0e4-7378-429b-a212-431f984561db") : configmap "kube-rbac-proxy" not found Jul 02 19:06:08.403303 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:08.403183 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s5ft7_7ac120d5-4418-44db-adea-9c186a1a46af/serve-healthcheck-canary/0.log" Jul 02 19:06:08.419526 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:08.419494 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:10.047042 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:10.047006 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-82vw5" event={"ID":"975ecd23-87b4-4897-b7c4-deeaa4487331","Type":"ContainerStarted","Data":"5de3c9d349f6d4ad3766aaf6764b37a15950acede9a4e213a39300906efdad9b"} Jul 02 19:06:10.064466 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:10.064420 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-82vw5" podStartSLOduration=1.831727479 podStartE2EDuration="4.064405068s" totalImagesPullingTime="2.232677589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:10.063747883 +0000 UTC m=+49.990756596" watchObservedRunningTime="2026-07-02 19:06:10.064405068 +0000 UTC m=+49.991413780" Jul 02 19:06:10.754932 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:10.754900 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:11.159768 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:11.159657 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c79663f-9a62-40e4-b870-f2f156b654d0-metrics-tls\") pod \"dns-default-66wg5\" (UID: \"3c79663f-9a62-40e4-b870-f2f156b654d0\") " pod="openshift-dns/dns-default-66wg5" Jul 02 19:06:11.160198 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:11.159803 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 19:06:11.160198 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:11.159887 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c79663f-9a62-40e4-b870-f2f156b654d0-metrics-tls podName:3c79663f-9a62-40e4-b870-f2f156b654d0 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:27.159861943 +0000 UTC m=+67.086870644 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c79663f-9a62-40e4-b870-f2f156b654d0-metrics-tls") pod "dns-default-66wg5" (UID: "3c79663f-9a62-40e4-b870-f2f156b654d0") : secret "dns-default-metrics-tls" not found Jul 02 19:06:11.354260 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:11.354208 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:11.587819 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:11.587783 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:11.689050 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:11.689005 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:11.793082 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:11.793038 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:11.958647 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:11.958556 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:12.251391 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:12.251363 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:12.267630 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:12.267604 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d13f0e4-7378-429b-a212-431f984561db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:12.267738 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:12.267646 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6d13f0e4-7378-429b-a212-431f984561db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:12.267776 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:12.267757 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 19:06:12.267843 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:12.267821 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d13f0e4-7378-429b-a212-431f984561db-insights-runtime-extractor-tls podName:6d13f0e4-7378-429b-a212-431f984561db nodeName:}" failed. No retries permitted until 2026-07-02 19:06:20.26780207 +0000 UTC m=+60.194810765 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6d13f0e4-7378-429b-a212-431f984561db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-d29gg" (UID: "6d13f0e4-7378-429b-a212-431f984561db") : secret "insights-runtime-extractor-tls" not found Jul 02 19:06:12.268162 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:12.268145 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6d13f0e4-7378-429b-a212-431f984561db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:12.634021 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:12.633909 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:13.253212 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:13.253166 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:13.828599 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:13.828564 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:14.047446 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:14.047396 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:14.231834 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:14.231801 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:14.395970 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:14.395935 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:14.496212 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:14.496115 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:14.655573 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:14.655531 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:15.027905 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:15.027868 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:15.164600 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:15.164565 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:15.433960 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:15.433836 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:15.534507 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:15.534473 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:15.635053 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:15.634998 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:15.691633 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:15.691523 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:15.844255 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:15.844224 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:15.906833 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:15.906801 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:16.042375 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:16.042337 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:17.045594 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:17.045561 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:17.426789 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:17.426710 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:17.660492 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:17.660459 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:17.727483 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:17.727455 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:17.824900 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:17.824865 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:18.029143 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:18.029056 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:19.028647 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:19.028617 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:19.204601 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:19.204567 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:19.262117 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:19.262092 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:19.629269 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:19.629232 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:19.823603 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:19.823561 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:20.229001 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:20.228967 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:20.326978 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:20.326940 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d13f0e4-7378-429b-a212-431f984561db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:20.329389 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:20.329358 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6d13f0e4-7378-429b-a212-431f984561db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-d29gg\" (UID: \"6d13f0e4-7378-429b-a212-431f984561db\") " pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:20.371411 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:20.371376 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-d29gg" Jul 02 19:06:20.522218 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:20.522149 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-d29gg"] Jul 02 19:06:20.525439 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:06:20.525416 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d13f0e4_7378_429b_a212_431f984561db.slice/crio-66be9973c43df1b20d1cfc99679fa741e13ba6c878afafb5ca60a44056ce3c49 WatchSource:0}: Error finding container 66be9973c43df1b20d1cfc99679fa741e13ba6c878afafb5ca60a44056ce3c49: Status 404 returned error can't find the container with id 66be9973c43df1b20d1cfc99679fa741e13ba6c878afafb5ca60a44056ce3c49 Jul 02 19:06:20.628807 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:20.628778 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:21.000217 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:21.000188 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-29hfm" Jul 02 19:06:21.075499 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:21.075454 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d29gg" event={"ID":"6d13f0e4-7378-429b-a212-431f984561db","Type":"ContainerStarted","Data":"66be9973c43df1b20d1cfc99679fa741e13ba6c878afafb5ca60a44056ce3c49"} Jul 02 19:06:21.431136 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:21.431052 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:21.638749 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:21.638714 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:21.837067 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:21.837029 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:22.032993 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:22.032956 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:25.088997 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:25.088953 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d29gg" event={"ID":"6d13f0e4-7378-429b-a212-431f984561db","Type":"ContainerStarted","Data":"d8964c852158ff7bf7263f7d14d16a4c870933010e5a82afe99750c53fb6ea96"} Jul 02 19:06:25.089357 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:25.089016 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d29gg" event={"ID":"6d13f0e4-7378-429b-a212-431f984561db","Type":"ContainerStarted","Data":"a9a16b42634e027d0d1f30ed367a5d77ac9b066a4fc622eba9999561297199c9"} Jul 02 19:06:26.008368 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.008329 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68d8b6d74-sfcjw"] Jul 02 19:06:26.010393 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.010370 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68d8b6d74-sfcjw" Jul 02 19:06:26.015038 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.015015 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 02 19:06:26.015721 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.015701 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-4wch5\"" Jul 02 19:06:26.015981 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.015969 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 02 19:06:26.016042 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.015979 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 02 19:06:26.017342 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.017323 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c76fbfc4-qwdb4"] Jul 02 19:06:26.019278 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.019258 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c76fbfc4-qwdb4" Jul 02 19:06:26.022036 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.022019 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 02 19:06:26.022483 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.022467 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 02 19:06:26.046283 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.046255 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c76fbfc4-qwdb4"] Jul 02 19:06:26.048090 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.048068 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68d8b6d74-sfcjw"] Jul 02 19:06:26.070551 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.070528 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0af0cf0f-7598-4a9b-8166-34e391cc133b-klusterlet-config\") pod \"klusterlet-addon-workmgr-66c76fbfc4-qwdb4\" (UID: \"0af0cf0f-7598-4a9b-8166-34e391cc133b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c76fbfc4-qwdb4" Jul 02 19:06:26.070710 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.070558 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d3733add-6a48-4a0d-87d9-f005d260b066-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68d8b6d74-sfcjw\" (UID: \"d3733add-6a48-4a0d-87d9-f005d260b066\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68d8b6d74-sfcjw" Jul 02 19:06:26.070710 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.070636 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtmjp\" (UniqueName: \"kubernetes.io/projected/d3733add-6a48-4a0d-87d9-f005d260b066-kube-api-access-jtmjp\") pod \"managed-serviceaccount-addon-agent-68d8b6d74-sfcjw\" (UID: \"d3733add-6a48-4a0d-87d9-f005d260b066\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68d8b6d74-sfcjw" Jul 02 19:06:26.070710 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.070690 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0af0cf0f-7598-4a9b-8166-34e391cc133b-tmp\") pod \"klusterlet-addon-workmgr-66c76fbfc4-qwdb4\" (UID: \"0af0cf0f-7598-4a9b-8166-34e391cc133b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c76fbfc4-qwdb4" Jul 02 19:06:26.070878 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.070763 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhfbc\" (UniqueName: \"kubernetes.io/projected/0af0cf0f-7598-4a9b-8166-34e391cc133b-kube-api-access-vhfbc\") pod \"klusterlet-addon-workmgr-66c76fbfc4-qwdb4\" (UID: \"0af0cf0f-7598-4a9b-8166-34e391cc133b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c76fbfc4-qwdb4" Jul 02 19:06:26.172110 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.172076 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhfbc\" (UniqueName: \"kubernetes.io/projected/0af0cf0f-7598-4a9b-8166-34e391cc133b-kube-api-access-vhfbc\") pod \"klusterlet-addon-workmgr-66c76fbfc4-qwdb4\" (UID: \"0af0cf0f-7598-4a9b-8166-34e391cc133b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c76fbfc4-qwdb4" Jul 02 19:06:26.172533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.172144 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0af0cf0f-7598-4a9b-8166-34e391cc133b-klusterlet-config\") pod \"klusterlet-addon-workmgr-66c76fbfc4-qwdb4\" (UID: \"0af0cf0f-7598-4a9b-8166-34e391cc133b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c76fbfc4-qwdb4" Jul 02 19:06:26.172533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.172174 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d3733add-6a48-4a0d-87d9-f005d260b066-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68d8b6d74-sfcjw\" (UID: \"d3733add-6a48-4a0d-87d9-f005d260b066\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68d8b6d74-sfcjw" Jul 02 19:06:26.172533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.172229 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtmjp\" (UniqueName: \"kubernetes.io/projected/d3733add-6a48-4a0d-87d9-f005d260b066-kube-api-access-jtmjp\") pod \"managed-serviceaccount-addon-agent-68d8b6d74-sfcjw\" (UID: \"d3733add-6a48-4a0d-87d9-f005d260b066\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68d8b6d74-sfcjw" Jul 02 19:06:26.172533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.172253 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0af0cf0f-7598-4a9b-8166-34e391cc133b-tmp\") pod \"klusterlet-addon-workmgr-66c76fbfc4-qwdb4\" (UID: \"0af0cf0f-7598-4a9b-8166-34e391cc133b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c76fbfc4-qwdb4" Jul 02 19:06:26.172878 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.172702 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0af0cf0f-7598-4a9b-8166-34e391cc133b-tmp\") pod \"klusterlet-addon-workmgr-66c76fbfc4-qwdb4\" (UID: \"0af0cf0f-7598-4a9b-8166-34e391cc133b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c76fbfc4-qwdb4" Jul 02 19:06:26.174735 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.174713 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0af0cf0f-7598-4a9b-8166-34e391cc133b-klusterlet-config\") pod \"klusterlet-addon-workmgr-66c76fbfc4-qwdb4\" (UID: \"0af0cf0f-7598-4a9b-8166-34e391cc133b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c76fbfc4-qwdb4" Jul 02 19:06:26.174839 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.174765 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d3733add-6a48-4a0d-87d9-f005d260b066-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68d8b6d74-sfcjw\" (UID: \"d3733add-6a48-4a0d-87d9-f005d260b066\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68d8b6d74-sfcjw" Jul 02 19:06:26.176281 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.176252 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:26.199620 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.199591 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtmjp\" (UniqueName: \"kubernetes.io/projected/d3733add-6a48-4a0d-87d9-f005d260b066-kube-api-access-jtmjp\") pod \"managed-serviceaccount-addon-agent-68d8b6d74-sfcjw\" (UID: \"d3733add-6a48-4a0d-87d9-f005d260b066\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68d8b6d74-sfcjw" Jul 02 19:06:26.199743 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.199719 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhfbc\" (UniqueName: \"kubernetes.io/projected/0af0cf0f-7598-4a9b-8166-34e391cc133b-kube-api-access-vhfbc\") pod \"klusterlet-addon-workmgr-66c76fbfc4-qwdb4\" (UID: \"0af0cf0f-7598-4a9b-8166-34e391cc133b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c76fbfc4-qwdb4" Jul 02 19:06:26.297079 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.296993 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:26.328658 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.328623 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68d8b6d74-sfcjw" Jul 02 19:06:26.336832 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.336806 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c76fbfc4-qwdb4" Jul 02 19:06:26.393976 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.393700 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:26.476000 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.475964 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs\") pod \"network-metrics-daemon-c9b6h\" (UID: \"92620d26-b178-4330-a011-98af64b4d43c\") " pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:06:26.477563 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.477535 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68d8b6d74-sfcjw"] Jul 02 19:06:26.478959 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.478932 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 19:06:26.483920 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.483898 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:26.489116 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.489089 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92620d26-b178-4330-a011-98af64b4d43c-metrics-certs\") pod \"network-metrics-daemon-c9b6h\" (UID: \"92620d26-b178-4330-a011-98af64b4d43c\") " pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:06:26.496143 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.496117 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c76fbfc4-qwdb4"] Jul 02 19:06:26.567325 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.567243 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jjgvz\"" Jul 02 19:06:26.575436 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.575412 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c9b6h" Jul 02 19:06:26.576822 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.576797 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret\") pod \"global-pull-secret-syncer-txtgt\" (UID: \"0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef\") " pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:06:26.576964 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.576852 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhqhb\" (UniqueName: \"kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb\") pod \"network-check-target-hssx4\" (UID: \"4d87225d-39f9-40d0-8ae7-5fa11099c16c\") " pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:06:26.580156 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.580133 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 19:06:26.580264 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.580133 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 19:06:26.589799 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.589777 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 19:06:26.589913 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.589773 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef-original-pull-secret\") pod \"global-pull-secret-syncer-txtgt\" (UID: \"0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef\") " pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:06:26.599915 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.599888 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhqhb\" (UniqueName: \"kubernetes.io/projected/4d87225d-39f9-40d0-8ae7-5fa11099c16c-kube-api-access-xhqhb\") pod \"network-check-target-hssx4\" (UID: \"4d87225d-39f9-40d0-8ae7-5fa11099c16c\") " pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:06:26.853575 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.853501 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mxlv5\"" Jul 02 19:06:26.858583 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.858559 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txtgt" Jul 02 19:06:26.861774 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:26.861740 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:06:27.182749 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:27.182600 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c79663f-9a62-40e4-b870-f2f156b654d0-metrics-tls\") pod \"dns-default-66wg5\" (UID: \"3c79663f-9a62-40e4-b870-f2f156b654d0\") " pod="openshift-dns/dns-default-66wg5" Jul 02 19:06:27.185298 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:27.185274 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c79663f-9a62-40e4-b870-f2f156b654d0-metrics-tls\") pod \"dns-default-66wg5\" (UID: \"3c79663f-9a62-40e4-b870-f2f156b654d0\") " pod="openshift-dns/dns-default-66wg5" Jul 02 19:06:27.480724 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:27.480692 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-crw8l\"" Jul 02 19:06:27.488790 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:27.488763 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-66wg5" Jul 02 19:06:27.549646 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:06:27.549616 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3733add_6a48_4a0d_87d9_f005d260b066.slice/crio-0e8f38040f444d727bfdb3c3e5a0628abefb21462ca1134fa601dfbcabbcb59f WatchSource:0}: Error finding container 0e8f38040f444d727bfdb3c3e5a0628abefb21462ca1134fa601dfbcabbcb59f: Status 404 returned error can't find the container with id 0e8f38040f444d727bfdb3c3e5a0628abefb21462ca1134fa601dfbcabbcb59f Jul 02 19:06:27.550619 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:06:27.550593 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0af0cf0f_7598_4a9b_8166_34e391cc133b.slice/crio-50569cc36adbc5161bff57999e3d97ade3c3b682fcd01e796d8c92d6cabe54e0 WatchSource:0}: Error finding container 50569cc36adbc5161bff57999e3d97ade3c3b682fcd01e796d8c92d6cabe54e0: Status 404 returned error can't find the container with id 50569cc36adbc5161bff57999e3d97ade3c3b682fcd01e796d8c92d6cabe54e0 Jul 02 19:06:27.706518 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:27.706482 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c9b6h"] Jul 02 19:06:27.727227 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:27.727195 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hssx4"] Jul 02 19:06:27.748558 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:27.748485 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:27.805648 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:06:27.805616 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92620d26_b178_4330_a011_98af64b4d43c.slice/crio-0db85de22f01f665bfbfb357d601ad64b0aea2784d4b67bc93083f405fbf7de1 WatchSource:0}: Error finding container 0db85de22f01f665bfbfb357d601ad64b0aea2784d4b67bc93083f405fbf7de1: Status 404 returned error can't find the container with id 0db85de22f01f665bfbfb357d601ad64b0aea2784d4b67bc93083f405fbf7de1 Jul 02 19:06:27.806167 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:06:27.806147 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d87225d_39f9_40d0_8ae7_5fa11099c16c.slice/crio-67a28ce1e8c8d2714061ea6540bb77253ca911c3d6e1d2a7417be322c18a740b WatchSource:0}: Error finding container 67a28ce1e8c8d2714061ea6540bb77253ca911c3d6e1d2a7417be322c18a740b: Status 404 returned error can't find the container with id 67a28ce1e8c8d2714061ea6540bb77253ca911c3d6e1d2a7417be322c18a740b Jul 02 19:06:27.832633 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:27.832608 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:27.934550 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:27.934524 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:27.944424 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:27.944395 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-txtgt"] Jul 02 19:06:27.947770 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:06:27.947734 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e7a0e42_4fee_4788_ba6a_4b7de6ed04ef.slice/crio-1d4e6a25b8865368cb7ead6230cf699d52ec7e28f35e69af583650bde66640e4 WatchSource:0}: Error finding container 1d4e6a25b8865368cb7ead6230cf699d52ec7e28f35e69af583650bde66640e4: Status 404 returned error can't find the container with id 1d4e6a25b8865368cb7ead6230cf699d52ec7e28f35e69af583650bde66640e4 Jul 02 19:06:27.947894 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:27.947769 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-66wg5"] Jul 02 19:06:27.951464 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:06:27.951443 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c79663f_9a62_40e4_b870_f2f156b654d0.slice/crio-7d4dc2fc00cca98eacb9ab47785b497c35b23fadecef2bbaac12777cddaedc42 WatchSource:0}: Error finding container 7d4dc2fc00cca98eacb9ab47785b497c35b23fadecef2bbaac12777cddaedc42: Status 404 returned error can't find the container with id 7d4dc2fc00cca98eacb9ab47785b497c35b23fadecef2bbaac12777cddaedc42 Jul 02 19:06:28.100171 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:28.100091 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-66wg5" event={"ID":"3c79663f-9a62-40e4-b870-f2f156b654d0","Type":"ContainerStarted","Data":"7d4dc2fc00cca98eacb9ab47785b497c35b23fadecef2bbaac12777cddaedc42"} Jul 02 19:06:28.101222 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:28.101190 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hssx4" event={"ID":"4d87225d-39f9-40d0-8ae7-5fa11099c16c","Type":"ContainerStarted","Data":"67a28ce1e8c8d2714061ea6540bb77253ca911c3d6e1d2a7417be322c18a740b"} Jul 02 19:06:28.103167 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:28.103144 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-d29gg" event={"ID":"6d13f0e4-7378-429b-a212-431f984561db","Type":"ContainerStarted","Data":"d2cdd4bc320fc3e342423b3d376a03df52e8004774b1c9c2c46a0bdfa4391b90"} Jul 02 19:06:28.104267 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:28.104246 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c9b6h" event={"ID":"92620d26-b178-4330-a011-98af64b4d43c","Type":"ContainerStarted","Data":"0db85de22f01f665bfbfb357d601ad64b0aea2784d4b67bc93083f405fbf7de1"} Jul 02 19:06:28.105215 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:28.105190 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c76fbfc4-qwdb4" event={"ID":"0af0cf0f-7598-4a9b-8166-34e391cc133b","Type":"ContainerStarted","Data":"50569cc36adbc5161bff57999e3d97ade3c3b682fcd01e796d8c92d6cabe54e0"} Jul 02 19:06:28.106208 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:28.106188 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-txtgt" event={"ID":"0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef","Type":"ContainerStarted","Data":"1d4e6a25b8865368cb7ead6230cf699d52ec7e28f35e69af583650bde66640e4"} Jul 02 19:06:28.107245 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:28.107225 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68d8b6d74-sfcjw" event={"ID":"d3733add-6a48-4a0d-87d9-f005d260b066","Type":"ContainerStarted","Data":"0e8f38040f444d727bfdb3c3e5a0628abefb21462ca1134fa601dfbcabbcb59f"} Jul 02 19:06:28.120188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:28.120005 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-d29gg" podStartSLOduration=16.917544252 podStartE2EDuration="24.119971041s" totalImagesPullingTime="7.202426789s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:06:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:28.119958514 +0000 UTC m=+68.046967242" watchObservedRunningTime="2026-07-02 19:06:28.119971041 +0000 UTC m=+68.046979752" Jul 02 19:06:29.391711 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:29.391661 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:29.459306 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:29.459195 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:33.042215 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:33.042173 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:33.113204 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:33.113166 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:33.200445 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:33.200413 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:33.270801 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:33.270767 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:33.345650 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:33.345568 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:33.492277 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:33.492242 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:33.587950 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:33.587914 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:33.656016 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:33.655927 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:33.774198 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:33.774163 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:34.185779 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:34.185746 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:34.271524 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:34.271490 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:34.369569 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:34.369487 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:34.576426 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:34.576389 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:35.167894 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:35.167863 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:35.369569 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:35.369536 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:35.986931 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:35.986901 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:36.057155 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.057118 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:36.146270 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.146233 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c9b6h" event={"ID":"92620d26-b178-4330-a011-98af64b4d43c","Type":"ContainerStarted","Data":"9c8ea8f3018dc624f0becf4c66c3f14bd2e5daa93dc2a2783c9ecf328b0e4d22"} Jul 02 19:06:36.146463 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.146328 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c9b6h" event={"ID":"92620d26-b178-4330-a011-98af64b4d43c","Type":"ContainerStarted","Data":"8cd63967315284e1e5e5e1f127489e0ca957cccc5e511cafe13498292c4c81e8"} Jul 02 19:06:36.147704 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.147664 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c76fbfc4-qwdb4" event={"ID":"0af0cf0f-7598-4a9b-8166-34e391cc133b","Type":"ContainerStarted","Data":"9e6c2eaad4455dbeebec977c9bd9e1a40bc22521a4da5af20a7b4f36c9c1b389"} Jul 02 19:06:36.147898 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.147874 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c76fbfc4-qwdb4" Jul 02 19:06:36.149349 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.149322 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-txtgt" event={"ID":"0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef","Type":"ContainerStarted","Data":"2ff758010fb452aa4a8f1721c89c04cbb7e6662e4ed89fd6a2bb69ca8a24a9ee"} Jul 02 19:06:36.149651 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.149636 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c76fbfc4-qwdb4" Jul 02 19:06:36.150979 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.150960 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68d8b6d74-sfcjw" event={"ID":"d3733add-6a48-4a0d-87d9-f005d260b066","Type":"ContainerStarted","Data":"418c46f8b4c062825e0ea6323ee43ce7b0423fc04886d39640992c282f520491"} Jul 02 19:06:36.152726 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.152703 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-66wg5" event={"ID":"3c79663f-9a62-40e4-b870-f2f156b654d0","Type":"ContainerStarted","Data":"b8b184fe76f453f35177a7f40282980b6530e0aab8fa6e9e40e4a251b9a50bea"} Jul 02 19:06:36.152833 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.152751 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-66wg5" event={"ID":"3c79663f-9a62-40e4-b870-f2f156b654d0","Type":"ContainerStarted","Data":"b52d420ad2389e6b0a0cc761158496bca5af5bb7a00d27090219a181e5108dd6"} Jul 02 19:06:36.152833 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.152784 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-66wg5" Jul 02 19:06:36.154150 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.154131 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hssx4" event={"ID":"4d87225d-39f9-40d0-8ae7-5fa11099c16c","Type":"ContainerStarted","Data":"893ef99cdb55988950ec54197b4ca3b0e73029bacc6713fde9e18a8cc23fecd1"} Jul 02 19:06:36.154266 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.154242 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:06:36.164632 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.164592 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-c9b6h" podStartSLOduration=74.465406744 podStartE2EDuration="1m16.164578352s" totalImagesPullingTime="1.699171608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:36.16359979 +0000 UTC m=+76.090608502" watchObservedRunningTime="2026-07-02 19:06:36.164578352 +0000 UTC m=+76.091587078" Jul 02 19:06:36.178421 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.178380 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68d8b6d74-sfcjw" podStartSLOduration=3.257659172 podStartE2EDuration="11.178367628s" totalImagesPullingTime="7.920708456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:06:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:36.177887472 +0000 UTC m=+76.104896196" watchObservedRunningTime="2026-07-02 19:06:36.178367628 +0000 UTC m=+76.105376340" Jul 02 19:06:36.200072 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.200023 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hssx4" podStartSLOduration=68.544441662 podStartE2EDuration="1m16.200007835s" totalImagesPullingTime="7.655566173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:36.199048694 +0000 UTC m=+76.126057407" watchObservedRunningTime="2026-07-02 19:06:36.200007835 +0000 UTC m=+76.127016546" Jul 02 19:06:36.215165 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.215107 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c76fbfc4-qwdb4" podStartSLOduration=3.285894825 podStartE2EDuration="11.215086708s" totalImagesPullingTime="7.929191883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:06:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:36.21415433 +0000 UTC m=+76.141163043" watchObservedRunningTime="2026-07-02 19:06:36.215086708 +0000 UTC m=+76.142095424" Jul 02 19:06:36.215930 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.215909 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:36.228401 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.228351 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-txtgt" podStartSLOduration=67.691037317 podStartE2EDuration="1m15.228333371s" totalImagesPullingTime="7.537296054s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:05:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:36.228221038 +0000 UTC m=+76.155229751" watchObservedRunningTime="2026-07-02 19:06:36.228333371 +0000 UTC m=+76.155342084" Jul 02 19:06:36.245298 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.245219 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-66wg5" podStartSLOduration=33.747051116 podStartE2EDuration="41.24520493s" totalImagesPullingTime="7.498153814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:05:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:36.244544605 +0000 UTC m=+76.171553318" watchObservedRunningTime="2026-07-02 19:06:36.24520493 +0000 UTC m=+76.172213641" Jul 02 19:06:36.263003 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.262982 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl"] Jul 02 19:06:36.265802 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.265785 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl" Jul 02 19:06:36.268218 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.268201 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 02 19:06:36.268350 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.268320 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 02 19:06:36.268431 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.268370 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 02 19:06:36.268431 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.268381 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 02 19:06:36.268431 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.268401 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 02 19:06:36.268578 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.268507 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-s474q\"" Jul 02 19:06:36.277286 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.277269 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl"] Jul 02 19:06:36.298594 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.298571 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:36.355214 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.355183 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:36.363287 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.363267 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6be4e678-7208-42b4-bf96-b6fd4bd121c4-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-dcvwl\" (UID: \"6be4e678-7208-42b4-bf96-b6fd4bd121c4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl" Jul 02 19:06:36.363361 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.363328 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6be4e678-7208-42b4-bf96-b6fd4bd121c4-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-dcvwl\" (UID: \"6be4e678-7208-42b4-bf96-b6fd4bd121c4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl" Jul 02 19:06:36.363436 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.363421 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6be4e678-7208-42b4-bf96-b6fd4bd121c4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-dcvwl\" (UID: \"6be4e678-7208-42b4-bf96-b6fd4bd121c4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl" Jul 02 19:06:36.363469 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.363448 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwvg5\" (UniqueName: \"kubernetes.io/projected/6be4e678-7208-42b4-bf96-b6fd4bd121c4-kube-api-access-xwvg5\") pod \"prometheus-operator-567f7b8b7c-dcvwl\" (UID: \"6be4e678-7208-42b4-bf96-b6fd4bd121c4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl" Jul 02 19:06:36.464593 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.464562 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6be4e678-7208-42b4-bf96-b6fd4bd121c4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-dcvwl\" (UID: \"6be4e678-7208-42b4-bf96-b6fd4bd121c4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl" Jul 02 19:06:36.465009 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.464598 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwvg5\" (UniqueName: \"kubernetes.io/projected/6be4e678-7208-42b4-bf96-b6fd4bd121c4-kube-api-access-xwvg5\") pod \"prometheus-operator-567f7b8b7c-dcvwl\" (UID: \"6be4e678-7208-42b4-bf96-b6fd4bd121c4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl" Jul 02 19:06:36.465009 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.464617 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6be4e678-7208-42b4-bf96-b6fd4bd121c4-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-dcvwl\" (UID: \"6be4e678-7208-42b4-bf96-b6fd4bd121c4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl" Jul 02 19:06:36.465009 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.464817 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6be4e678-7208-42b4-bf96-b6fd4bd121c4-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-dcvwl\" (UID: \"6be4e678-7208-42b4-bf96-b6fd4bd121c4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl" Jul 02 19:06:36.465009 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:36.464957 2566 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 02 19:06:36.465191 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:36.465063 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6be4e678-7208-42b4-bf96-b6fd4bd121c4-prometheus-operator-tls podName:6be4e678-7208-42b4-bf96-b6fd4bd121c4 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:36.965045157 +0000 UTC m=+76.892053847 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/6be4e678-7208-42b4-bf96-b6fd4bd121c4-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-dcvwl" (UID: "6be4e678-7208-42b4-bf96-b6fd4bd121c4") : secret "prometheus-operator-tls" not found Jul 02 19:06:36.465397 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.465379 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6be4e678-7208-42b4-bf96-b6fd4bd121c4-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-dcvwl\" (UID: \"6be4e678-7208-42b4-bf96-b6fd4bd121c4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl" Jul 02 19:06:36.467036 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.467013 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6be4e678-7208-42b4-bf96-b6fd4bd121c4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-dcvwl\" (UID: \"6be4e678-7208-42b4-bf96-b6fd4bd121c4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl" Jul 02 19:06:36.480340 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.480318 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwvg5\" (UniqueName: \"kubernetes.io/projected/6be4e678-7208-42b4-bf96-b6fd4bd121c4-kube-api-access-xwvg5\") pod \"prometheus-operator-567f7b8b7c-dcvwl\" (UID: \"6be4e678-7208-42b4-bf96-b6fd4bd121c4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl" Jul 02 19:06:36.634746 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.634633 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:36.968891 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.968801 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6be4e678-7208-42b4-bf96-b6fd4bd121c4-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-dcvwl\" (UID: \"6be4e678-7208-42b4-bf96-b6fd4bd121c4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl" Jul 02 19:06:36.971160 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:36.971130 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/6be4e678-7208-42b4-bf96-b6fd4bd121c4-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-dcvwl\" (UID: \"6be4e678-7208-42b4-bf96-b6fd4bd121c4\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl" Jul 02 19:06:37.174965 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:37.174931 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl" Jul 02 19:06:37.294946 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:37.294910 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl"] Jul 02 19:06:37.297561 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:06:37.297539 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6be4e678_7208_42b4_bf96_b6fd4bd121c4.slice/crio-c14074e857d25ea39b2dc3884b46ba5d88060d9224fbf1bc7534e389e1addb29 WatchSource:0}: Error finding container c14074e857d25ea39b2dc3884b46ba5d88060d9224fbf1bc7534e389e1addb29: Status 404 returned error can't find the container with id c14074e857d25ea39b2dc3884b46ba5d88060d9224fbf1bc7534e389e1addb29 Jul 02 19:06:38.167414 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.167382 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl" event={"ID":"6be4e678-7208-42b4-bf96-b6fd4bd121c4","Type":"ContainerStarted","Data":"c14074e857d25ea39b2dc3884b46ba5d88060d9224fbf1bc7534e389e1addb29"} Jul 02 19:06:38.595057 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.595032 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57d75f5485-sg5kb"] Jul 02 19:06:38.599122 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.599098 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:38.602642 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.602607 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 02 19:06:38.602642 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.602627 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 02 19:06:38.602838 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.602645 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9hr2v\"" Jul 02 19:06:38.602838 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.602660 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 02 19:06:38.602838 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.602710 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 02 19:06:38.603026 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.603005 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 02 19:06:38.603123 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.603062 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 02 19:06:38.603123 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.603084 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 02 19:06:38.612547 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.612524 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57d75f5485-sg5kb"] Jul 02 19:06:38.679921 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.679888 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c55205c4-e3ed-44ac-89d7-12c11cb1913b-oauth-serving-cert\") pod \"console-57d75f5485-sg5kb\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:38.680092 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.679931 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c55205c4-e3ed-44ac-89d7-12c11cb1913b-service-ca\") pod \"console-57d75f5485-sg5kb\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:38.680092 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.679977 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j82g\" (UniqueName: \"kubernetes.io/projected/c55205c4-e3ed-44ac-89d7-12c11cb1913b-kube-api-access-9j82g\") pod \"console-57d75f5485-sg5kb\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:38.680092 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.680037 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c55205c4-e3ed-44ac-89d7-12c11cb1913b-console-config\") pod \"console-57d75f5485-sg5kb\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:38.680256 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.680118 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c55205c4-e3ed-44ac-89d7-12c11cb1913b-console-oauth-config\") pod \"console-57d75f5485-sg5kb\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:38.680256 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.680160 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c55205c4-e3ed-44ac-89d7-12c11cb1913b-console-serving-cert\") pod \"console-57d75f5485-sg5kb\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:38.760386 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.760294 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:38.781337 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.781310 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c55205c4-e3ed-44ac-89d7-12c11cb1913b-console-serving-cert\") pod \"console-57d75f5485-sg5kb\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:38.781426 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.781344 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c55205c4-e3ed-44ac-89d7-12c11cb1913b-oauth-serving-cert\") pod \"console-57d75f5485-sg5kb\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:38.781426 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.781365 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c55205c4-e3ed-44ac-89d7-12c11cb1913b-service-ca\") pod \"console-57d75f5485-sg5kb\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:38.781426 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.781388 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j82g\" (UniqueName: \"kubernetes.io/projected/c55205c4-e3ed-44ac-89d7-12c11cb1913b-kube-api-access-9j82g\") pod \"console-57d75f5485-sg5kb\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:38.781544 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.781434 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c55205c4-e3ed-44ac-89d7-12c11cb1913b-console-config\") pod \"console-57d75f5485-sg5kb\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:38.781544 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.781464 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c55205c4-e3ed-44ac-89d7-12c11cb1913b-console-oauth-config\") pod \"console-57d75f5485-sg5kb\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:38.782175 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.782150 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c55205c4-e3ed-44ac-89d7-12c11cb1913b-console-config\") pod \"console-57d75f5485-sg5kb\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:38.782297 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.782255 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c55205c4-e3ed-44ac-89d7-12c11cb1913b-oauth-serving-cert\") pod \"console-57d75f5485-sg5kb\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:38.782297 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.782276 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c55205c4-e3ed-44ac-89d7-12c11cb1913b-service-ca\") pod \"console-57d75f5485-sg5kb\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:38.783769 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.783743 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c55205c4-e3ed-44ac-89d7-12c11cb1913b-console-oauth-config\") pod \"console-57d75f5485-sg5kb\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:38.783879 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.783859 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c55205c4-e3ed-44ac-89d7-12c11cb1913b-console-serving-cert\") pod \"console-57d75f5485-sg5kb\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:38.790722 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.790671 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j82g\" (UniqueName: \"kubernetes.io/projected/c55205c4-e3ed-44ac-89d7-12c11cb1913b-kube-api-access-9j82g\") pod \"console-57d75f5485-sg5kb\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:38.908758 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:38.908716 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:39.035367 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:39.035289 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57d75f5485-sg5kb"] Jul 02 19:06:39.038272 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:06:39.038246 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc55205c4_e3ed_44ac_89d7_12c11cb1913b.slice/crio-975bb9096a11d18d37267e8d9ed67a8fba2e7de92f7c046186fd7480996c98c2 WatchSource:0}: Error finding container 975bb9096a11d18d37267e8d9ed67a8fba2e7de92f7c046186fd7480996c98c2: Status 404 returned error can't find the container with id 975bb9096a11d18d37267e8d9ed67a8fba2e7de92f7c046186fd7480996c98c2 Jul 02 19:06:39.172473 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:39.172436 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl" event={"ID":"6be4e678-7208-42b4-bf96-b6fd4bd121c4","Type":"ContainerStarted","Data":"ab81433afc11b552215208093fc272c31c69f6831916cf3b70e373c2e53188e3"} Jul 02 19:06:39.172916 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:39.172501 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl" event={"ID":"6be4e678-7208-42b4-bf96-b6fd4bd121c4","Type":"ContainerStarted","Data":"c92a5c9cd029b72b2df4ff38c9a7c3b6fe41779140668495cd029737b87c1e93"} Jul 02 19:06:39.173622 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:39.173597 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d75f5485-sg5kb" event={"ID":"c55205c4-e3ed-44ac-89d7-12c11cb1913b","Type":"ContainerStarted","Data":"975bb9096a11d18d37267e8d9ed67a8fba2e7de92f7c046186fd7480996c98c2"} Jul 02 19:06:39.189085 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:39.189037 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dcvwl" podStartSLOduration=1.998576173 podStartE2EDuration="3.189020848s" totalImagesPullingTime="1.190444675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:39.187609328 +0000 UTC m=+79.114618041" watchObservedRunningTime="2026-07-02 19:06:39.189020848 +0000 UTC m=+79.116029560" Jul 02 19:06:40.361248 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.361198 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:40.595117 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.595080 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:40.672982 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.671872 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hskfk"] Jul 02 19:06:40.683546 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.683517 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.686428 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.685598 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:40.686428 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.685980 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-p6znq\"" Jul 02 19:06:40.686428 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.686240 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 02 19:06:40.686662 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.686557 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 02 19:06:40.686846 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.686828 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 02 19:06:40.796595 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.796187 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a4adc67b-70b1-40cb-83f2-1d01aa96683e-node-exporter-tls\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.796595 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.796235 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4adc67b-70b1-40cb-83f2-1d01aa96683e-metrics-client-ca\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.796595 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.796266 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a4adc67b-70b1-40cb-83f2-1d01aa96683e-node-exporter-textfile\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.796595 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.796311 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4adc67b-70b1-40cb-83f2-1d01aa96683e-sys\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.796595 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.796342 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a4adc67b-70b1-40cb-83f2-1d01aa96683e-node-exporter-wtmp\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.796595 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.796367 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p27m\" (UniqueName: \"kubernetes.io/projected/a4adc67b-70b1-40cb-83f2-1d01aa96683e-kube-api-access-2p27m\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.796595 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.796397 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a4adc67b-70b1-40cb-83f2-1d01aa96683e-node-exporter-accelerators-collector-config\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.796595 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.796441 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4adc67b-70b1-40cb-83f2-1d01aa96683e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.796595 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.796464 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a4adc67b-70b1-40cb-83f2-1d01aa96683e-root\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.878529 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.878448 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:40.896860 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.896829 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4adc67b-70b1-40cb-83f2-1d01aa96683e-sys\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.897019 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.896881 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a4adc67b-70b1-40cb-83f2-1d01aa96683e-node-exporter-wtmp\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.897019 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.896908 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p27m\" (UniqueName: \"kubernetes.io/projected/a4adc67b-70b1-40cb-83f2-1d01aa96683e-kube-api-access-2p27m\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.897019 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.896922 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4adc67b-70b1-40cb-83f2-1d01aa96683e-sys\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.897019 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.896937 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a4adc67b-70b1-40cb-83f2-1d01aa96683e-node-exporter-accelerators-collector-config\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.897019 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.896982 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4adc67b-70b1-40cb-83f2-1d01aa96683e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.897019 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.897005 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a4adc67b-70b1-40cb-83f2-1d01aa96683e-root\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.897312 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.897038 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a4adc67b-70b1-40cb-83f2-1d01aa96683e-node-exporter-tls\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.897312 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.897062 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4adc67b-70b1-40cb-83f2-1d01aa96683e-metrics-client-ca\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.897312 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.897079 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a4adc67b-70b1-40cb-83f2-1d01aa96683e-node-exporter-wtmp\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.897312 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.897133 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a4adc67b-70b1-40cb-83f2-1d01aa96683e-root\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.897312 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:40.897202 2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 02 19:06:40.897312 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:40.897257 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4adc67b-70b1-40cb-83f2-1d01aa96683e-node-exporter-tls podName:a4adc67b-70b1-40cb-83f2-1d01aa96683e nodeName:}" failed. No retries permitted until 2026-07-02 19:06:41.397237597 +0000 UTC m=+81.324246296 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a4adc67b-70b1-40cb-83f2-1d01aa96683e-node-exporter-tls") pod "node-exporter-hskfk" (UID: "a4adc67b-70b1-40cb-83f2-1d01aa96683e") : secret "node-exporter-tls" not found Jul 02 19:06:40.897668 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.897614 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a4adc67b-70b1-40cb-83f2-1d01aa96683e-node-exporter-accelerators-collector-config\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.897668 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.897086 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a4adc67b-70b1-40cb-83f2-1d01aa96683e-node-exporter-textfile\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.898070 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.897939 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4adc67b-70b1-40cb-83f2-1d01aa96683e-metrics-client-ca\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.898070 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.897939 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a4adc67b-70b1-40cb-83f2-1d01aa96683e-node-exporter-textfile\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.910761 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.910730 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4adc67b-70b1-40cb-83f2-1d01aa96683e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.913070 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.913027 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p27m\" (UniqueName: \"kubernetes.io/projected/a4adc67b-70b1-40cb-83f2-1d01aa96683e-kube-api-access-2p27m\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:40.978740 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:40.978707 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:41.079412 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.079375 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:41.179274 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.179225 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:41.403011 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.402934 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a4adc67b-70b1-40cb-83f2-1d01aa96683e-node-exporter-tls\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:41.405538 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.405510 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a4adc67b-70b1-40cb-83f2-1d01aa96683e-node-exporter-tls\") pod \"node-exporter-hskfk\" (UID: \"a4adc67b-70b1-40cb-83f2-1d01aa96683e\") " pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:41.591059 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.591023 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:41.597721 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.597693 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hskfk" Jul 02 19:06:41.680934 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.680832 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:41.685455 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.685428 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 19:06:41.689322 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.689300 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.691699 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.691636 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 02 19:06:41.694533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.692376 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 02 19:06:41.694533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.692509 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 02 19:06:41.694533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.692589 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 02 19:06:41.694533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.692892 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 02 19:06:41.694533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.692984 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 02 19:06:41.694533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.693125 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 02 19:06:41.694533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.693458 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 02 19:06:41.694533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.693590 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 02 19:06:41.694533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.693982 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2cmkd\"" Jul 02 19:06:41.703789 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.703769 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 19:06:41.806368 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.806327 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.806560 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.806425 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m78fq\" (UniqueName: \"kubernetes.io/projected/e5821ec4-122b-4c9b-b413-83c3e7b026c4-kube-api-access-m78fq\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.806560 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.806456 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-config-volume\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.806560 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.806483 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.806753 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.806550 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.806753 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.806595 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e5821ec4-122b-4c9b-b413-83c3e7b026c4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.806753 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.806653 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e5821ec4-122b-4c9b-b413-83c3e7b026c4-config-out\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.806753 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.806696 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.806753 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.806724 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-web-config\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.806753 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.806745 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e5821ec4-122b-4c9b-b413-83c3e7b026c4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.806955 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.806770 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e5821ec4-122b-4c9b-b413-83c3e7b026c4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.806955 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.806841 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.806955 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.806863 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5821ec4-122b-4c9b-b413-83c3e7b026c4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.907824 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.907781 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e5821ec4-122b-4c9b-b413-83c3e7b026c4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.908012 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.907870 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e5821ec4-122b-4c9b-b413-83c3e7b026c4-config-out\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.908012 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.907899 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.908012 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.907936 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-web-config\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.908012 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.907960 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e5821ec4-122b-4c9b-b413-83c3e7b026c4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.909547 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.908365 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e5821ec4-122b-4c9b-b413-83c3e7b026c4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.909547 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.908444 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.909547 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.908475 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5821ec4-122b-4c9b-b413-83c3e7b026c4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.909547 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.908507 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.909547 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.908561 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m78fq\" (UniqueName: \"kubernetes.io/projected/e5821ec4-122b-4c9b-b413-83c3e7b026c4-kube-api-access-m78fq\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.909547 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.908596 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-config-volume\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.909547 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:41.908597 2566 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 02 19:06:41.909547 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.908709 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e5821ec4-122b-4c9b-b413-83c3e7b026c4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.909547 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:41.908727 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-main-tls podName:e5821ec4-122b-4c9b-b413-83c3e7b026c4 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:42.408662792 +0000 UTC m=+82.335671488 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "e5821ec4-122b-4c9b-b413-83c3e7b026c4") : secret "alertmanager-main-tls" not found Jul 02 19:06:41.909547 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:41.909122 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e5821ec4-122b-4c9b-b413-83c3e7b026c4-alertmanager-trusted-ca-bundle podName:e5821ec4-122b-4c9b-b413-83c3e7b026c4 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:42.40910627 +0000 UTC m=+82.336114960 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/e5821ec4-122b-4c9b-b413-83c3e7b026c4-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "e5821ec4-122b-4c9b-b413-83c3e7b026c4") : configmap references non-existent config key: ca-bundle.crt Jul 02 19:06:41.909547 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.909159 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.909547 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.909193 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.909547 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.909244 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e5821ec4-122b-4c9b-b413-83c3e7b026c4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.910961 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.910933 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.911080 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.911034 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e5821ec4-122b-4c9b-b413-83c3e7b026c4-config-out\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.911417 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.911394 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e5821ec4-122b-4c9b-b413-83c3e7b026c4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.911809 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.911784 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-config-volume\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.912198 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.912177 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-web-config\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.912994 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.912966 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.913452 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.913426 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.913545 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.913436 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.920537 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:41.920513 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m78fq\" (UniqueName: \"kubernetes.io/projected/e5821ec4-122b-4c9b-b413-83c3e7b026c4-kube-api-access-m78fq\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:41.929931 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:06:41.929906 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4adc67b_70b1_40cb_83f2_1d01aa96683e.slice/crio-ee1ca368445d063a0961afa5b920f549c275f0137cd15c9ef2adf99396a4d595 WatchSource:0}: Error finding container ee1ca368445d063a0961afa5b920f549c275f0137cd15c9ef2adf99396a4d595: Status 404 returned error can't find the container with id ee1ca368445d063a0961afa5b920f549c275f0137cd15c9ef2adf99396a4d595 Jul 02 19:06:42.006178 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:42.006148 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:42.186214 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:42.186120 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hskfk" event={"ID":"a4adc67b-70b1-40cb-83f2-1d01aa96683e","Type":"ContainerStarted","Data":"ee1ca368445d063a0961afa5b920f549c275f0137cd15c9ef2adf99396a4d595"} Jul 02 19:06:42.187633 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:42.187600 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d75f5485-sg5kb" event={"ID":"c55205c4-e3ed-44ac-89d7-12c11cb1913b","Type":"ContainerStarted","Data":"720f95ad6087a33935ace5e56184af0bb6f6f0738d9fffb9cf46c0aa44f2d273"} Jul 02 19:06:42.208913 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:42.208865 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57d75f5485-sg5kb" podStartSLOduration=1.292666707 podStartE2EDuration="4.208851134s" totalImagesPullingTime="2.916184427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:06:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:42.207656913 +0000 UTC m=+82.134665628" watchObservedRunningTime="2026-07-02 19:06:42.208851134 +0000 UTC m=+82.135859846" Jul 02 19:06:42.412843 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:42.412796 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:42.413248 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:42.412850 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5821ec4-122b-4c9b-b413-83c3e7b026c4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:42.414236 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:42.414213 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5821ec4-122b-4c9b-b413-83c3e7b026c4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:42.415577 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:42.415555 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:42.602215 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:42.602181 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:06:42.883143 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:42.883120 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 19:06:42.885175 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:06:42.885146 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5821ec4_122b_4c9b_b413_83c3e7b026c4.slice/crio-329998fd0fe9e574a8e873603c097ae7ed4cde4554e3565812f718763c2f5197 WatchSource:0}: Error finding container 329998fd0fe9e574a8e873603c097ae7ed4cde4554e3565812f718763c2f5197: Status 404 returned error can't find the container with id 329998fd0fe9e574a8e873603c097ae7ed4cde4554e3565812f718763c2f5197 Jul 02 19:06:43.193135 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:43.193044 2566 generic.go:356] "Generic (PLEG): container finished" podID="a4adc67b-70b1-40cb-83f2-1d01aa96683e" containerID="045ab9e08e930bdd56d757548f0bf7eb6a60eee56cea0d1e6682b82d0dfc92f3" exitCode=0 Jul 02 19:06:43.193315 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:43.193138 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hskfk" event={"ID":"a4adc67b-70b1-40cb-83f2-1d01aa96683e","Type":"ContainerDied","Data":"045ab9e08e930bdd56d757548f0bf7eb6a60eee56cea0d1e6682b82d0dfc92f3"} Jul 02 19:06:43.194496 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:43.194462 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5821ec4-122b-4c9b-b413-83c3e7b026c4","Type":"ContainerStarted","Data":"329998fd0fe9e574a8e873603c097ae7ed4cde4554e3565812f718763c2f5197"} Jul 02 19:06:43.550862 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:43.550553 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:43.634569 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:43.634534 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:43.702812 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:43.702779 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:43.903837 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:43.903741 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:44.200221 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:44.200128 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hskfk" event={"ID":"a4adc67b-70b1-40cb-83f2-1d01aa96683e","Type":"ContainerStarted","Data":"10432534ddb833003a4370068a42fe1b945f027d493a58bd75cda2f357acbf6e"} Jul 02 19:06:44.200221 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:44.200191 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hskfk" event={"ID":"a4adc67b-70b1-40cb-83f2-1d01aa96683e","Type":"ContainerStarted","Data":"4d56235a5f5f1ab1e516c0d2be2ab5ad3a3442c2ca534d4b0f5bf2f95426c2b2"} Jul 02 19:06:44.201591 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:44.201563 2566 generic.go:356] "Generic (PLEG): container finished" podID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerID="cd1cf0e8c17d064eec3bf26389d8f0189db8e5488f5130dee82977c982579ec4" exitCode=0 Jul 02 19:06:44.201703 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:44.201610 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5821ec4-122b-4c9b-b413-83c3e7b026c4","Type":"ContainerDied","Data":"cd1cf0e8c17d064eec3bf26389d8f0189db8e5488f5130dee82977c982579ec4"} Jul 02 19:06:44.218666 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:44.218613 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hskfk" podStartSLOduration=3.308857362 podStartE2EDuration="4.218593303s" totalImagesPullingTime="843.614513ms" totalInitContainerRuntime="66.121428ms" isStatefulPod=true podCreationTimestamp="2026-07-02 19:06:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:44.218213973 +0000 UTC m=+84.145222690" watchObservedRunningTime="2026-07-02 19:06:44.218593303 +0000 UTC m=+84.145602017" Jul 02 19:06:44.399632 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:44.399601 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:44.460549 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:44.460473 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:44.804291 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:44.804254 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86c9f6454-b5php"] Jul 02 19:06:44.806393 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:44.806363 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:44.811538 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:44.811512 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:44.815829 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:44.815806 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 02 19:06:44.826477 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:44.826446 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86c9f6454-b5php"] Jul 02 19:06:44.874258 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:44.874224 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:44.934576 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:44.934542 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3febe46a-7653-486b-a07f-65bb190912c6-console-serving-cert\") pod \"console-86c9f6454-b5php\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:44.934755 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:44.934584 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-oauth-serving-cert\") pod \"console-86c9f6454-b5php\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:44.934755 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:44.934646 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3febe46a-7653-486b-a07f-65bb190912c6-console-oauth-config\") pod \"console-86c9f6454-b5php\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:44.934755 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:44.934713 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-console-config\") pod \"console-86c9f6454-b5php\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:44.934755 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:44.934737 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-service-ca\") pod \"console-86c9f6454-b5php\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:44.934755 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:44.934751 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6wpm\" (UniqueName: \"kubernetes.io/projected/3febe46a-7653-486b-a07f-65bb190912c6-kube-api-access-r6wpm\") pod \"console-86c9f6454-b5php\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:44.934946 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:44.934810 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-trusted-ca-bundle\") pod \"console-86c9f6454-b5php\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:45.035923 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.035888 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-trusted-ca-bundle\") pod \"console-86c9f6454-b5php\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:45.036126 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.035946 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3febe46a-7653-486b-a07f-65bb190912c6-console-serving-cert\") pod \"console-86c9f6454-b5php\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:45.036126 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.035973 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-oauth-serving-cert\") pod \"console-86c9f6454-b5php\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:45.036126 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.036008 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3febe46a-7653-486b-a07f-65bb190912c6-console-oauth-config\") pod \"console-86c9f6454-b5php\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:45.036126 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.036049 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-console-config\") pod \"console-86c9f6454-b5php\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:45.036126 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.036075 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-service-ca\") pod \"console-86c9f6454-b5php\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:45.036126 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.036099 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6wpm\" (UniqueName: \"kubernetes.io/projected/3febe46a-7653-486b-a07f-65bb190912c6-kube-api-access-r6wpm\") pod \"console-86c9f6454-b5php\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:45.036898 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.036870 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-trusted-ca-bundle\") pod \"console-86c9f6454-b5php\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:45.037024 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.036897 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-oauth-serving-cert\") pod \"console-86c9f6454-b5php\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:45.037024 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.036990 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-console-config\") pod \"console-86c9f6454-b5php\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:45.037134 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.037053 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-service-ca\") pod \"console-86c9f6454-b5php\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:45.038951 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.038931 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3febe46a-7653-486b-a07f-65bb190912c6-console-serving-cert\") pod \"console-86c9f6454-b5php\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:45.039232 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.039207 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3febe46a-7653-486b-a07f-65bb190912c6-console-oauth-config\") pod \"console-86c9f6454-b5php\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:45.044027 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.044002 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6wpm\" (UniqueName: \"kubernetes.io/projected/3febe46a-7653-486b-a07f-65bb190912c6-kube-api-access-r6wpm\") pod \"console-86c9f6454-b5php\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:45.118137 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.118050 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:45.267446 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.267412 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86c9f6454-b5php"] Jul 02 19:06:45.387547 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.387466 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:45.395894 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.395863 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-hrhd2"] Jul 02 19:06:45.398807 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.398771 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-hrhd2" Jul 02 19:06:45.401298 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.401277 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-z58dq\"" Jul 02 19:06:45.401583 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.401567 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 02 19:06:45.408279 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.407705 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-hrhd2"] Jul 02 19:06:45.462783 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.462737 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:45.540060 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.540024 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e09cfa31-6ce1-485a-9539-1817701d1e95-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-hrhd2\" (UID: \"e09cfa31-6ce1-485a-9539-1817701d1e95\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-hrhd2" Jul 02 19:06:45.540060 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.540064 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwxlq\" (UniqueName: \"kubernetes.io/projected/e09cfa31-6ce1-485a-9539-1817701d1e95-kube-api-access-nwxlq\") pod \"monitoring-plugin-d5dcff7cf-hrhd2\" (UID: \"e09cfa31-6ce1-485a-9539-1817701d1e95\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-hrhd2" Jul 02 19:06:45.587504 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:06:45.587470 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3febe46a_7653_486b_a07f_65bb190912c6.slice/crio-9b92f97f2040ce710657d980b2ffd54bd60daffdb3fdc11fdf184b6c326c9cc0 WatchSource:0}: Error finding container 9b92f97f2040ce710657d980b2ffd54bd60daffdb3fdc11fdf184b6c326c9cc0: Status 404 returned error can't find the container with id 9b92f97f2040ce710657d980b2ffd54bd60daffdb3fdc11fdf184b6c326c9cc0 Jul 02 19:06:45.640710 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.640669 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwxlq\" (UniqueName: \"kubernetes.io/projected/e09cfa31-6ce1-485a-9539-1817701d1e95-kube-api-access-nwxlq\") pod \"monitoring-plugin-d5dcff7cf-hrhd2\" (UID: \"e09cfa31-6ce1-485a-9539-1817701d1e95\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-hrhd2" Jul 02 19:06:45.640832 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.640802 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e09cfa31-6ce1-485a-9539-1817701d1e95-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-hrhd2\" (UID: \"e09cfa31-6ce1-485a-9539-1817701d1e95\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-hrhd2" Jul 02 19:06:45.640927 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:45.640912 2566 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 02 19:06:45.640995 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:06:45.640981 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e09cfa31-6ce1-485a-9539-1817701d1e95-monitoring-plugin-cert podName:e09cfa31-6ce1-485a-9539-1817701d1e95 nodeName:}" failed. No retries permitted until 2026-07-02 19:06:46.140958939 +0000 UTC m=+86.067967642 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/e09cfa31-6ce1-485a-9539-1817701d1e95-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-hrhd2" (UID: "e09cfa31-6ce1-485a-9539-1817701d1e95") : secret "monitoring-plugin-cert" not found Jul 02 19:06:45.649885 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.649866 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwxlq\" (UniqueName: \"kubernetes.io/projected/e09cfa31-6ce1-485a-9539-1817701d1e95-kube-api-access-nwxlq\") pod \"monitoring-plugin-d5dcff7cf-hrhd2\" (UID: \"e09cfa31-6ce1-485a-9539-1817701d1e95\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-hrhd2" Jul 02 19:06:45.869471 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:45.869410 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:46.145821 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:46.145708 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e09cfa31-6ce1-485a-9539-1817701d1e95-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-hrhd2\" (UID: \"e09cfa31-6ce1-485a-9539-1817701d1e95\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-hrhd2" Jul 02 19:06:46.148368 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:46.148336 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e09cfa31-6ce1-485a-9539-1817701d1e95-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-hrhd2\" (UID: \"e09cfa31-6ce1-485a-9539-1817701d1e95\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-hrhd2" Jul 02 19:06:46.163558 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:46.163535 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-66wg5" Jul 02 19:06:46.213039 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:46.213002 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5821ec4-122b-4c9b-b413-83c3e7b026c4","Type":"ContainerStarted","Data":"61fbcb9b07310d85695d98ac08c334ac96a1819791bbe8ebf2787ad02a64ed99"} Jul 02 19:06:46.213209 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:46.213088 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5821ec4-122b-4c9b-b413-83c3e7b026c4","Type":"ContainerStarted","Data":"a97c6b86814e47a3132c58b1c167a598550e9f47184893552b9c42cdadea50cd"} Jul 02 19:06:46.213209 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:46.213134 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5821ec4-122b-4c9b-b413-83c3e7b026c4","Type":"ContainerStarted","Data":"d783be1bcfaa3595f2185212a7057938130e7814baa580a089aa2d8bbda760c2"} Jul 02 19:06:46.213209 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:46.213174 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5821ec4-122b-4c9b-b413-83c3e7b026c4","Type":"ContainerStarted","Data":"a5f1074c73fba7ab554bc11188b7b04d14d3d17d9268fba500058ad3135cfe8b"} Jul 02 19:06:46.213369 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:46.213212 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5821ec4-122b-4c9b-b413-83c3e7b026c4","Type":"ContainerStarted","Data":"9ec02cecdc81bc54ad3d0caa7b4baa1c66b2a1f9dd30ea6f09c64feb96263931"} Jul 02 19:06:46.215023 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:46.214995 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c9f6454-b5php" event={"ID":"3febe46a-7653-486b-a07f-65bb190912c6","Type":"ContainerStarted","Data":"f76d1a94a966a2389583b3e673a9ded639c58cbf22cff231cff8d679122d2d9c"} Jul 02 19:06:46.215158 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:46.215038 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c9f6454-b5php" event={"ID":"3febe46a-7653-486b-a07f-65bb190912c6","Type":"ContainerStarted","Data":"9b92f97f2040ce710657d980b2ffd54bd60daffdb3fdc11fdf184b6c326c9cc0"} Jul 02 19:06:46.238597 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:46.238551 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86c9f6454-b5php" podStartSLOduration=2.238536613 podStartE2EDuration="2.238536613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:06:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:46.238315865 +0000 UTC m=+86.165324579" watchObservedRunningTime="2026-07-02 19:06:46.238536613 +0000 UTC m=+86.165545324" Jul 02 19:06:46.314415 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:46.314382 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-hrhd2" Jul 02 19:06:46.421556 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:46.421458 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:46.455147 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:46.455117 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-hrhd2"] Jul 02 19:06:46.458612 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:06:46.458580 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode09cfa31_6ce1_485a_9539_1817701d1e95.slice/crio-5ba9da02977e7d49d0051bd3f85476a7e664107e052a642d049d57e105d1b984 WatchSource:0}: Error finding container 5ba9da02977e7d49d0051bd3f85476a7e664107e052a642d049d57e105d1b984: Status 404 returned error can't find the container with id 5ba9da02977e7d49d0051bd3f85476a7e664107e052a642d049d57e105d1b984 Jul 02 19:06:46.588710 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:46.588641 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:46.688545 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:46.688439 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:46.884165 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:46.884131 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:47.123137 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:47.123096 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:47.224697 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:47.224648 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:47.225527 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:47.225495 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5821ec4-122b-4c9b-b413-83c3e7b026c4","Type":"ContainerStarted","Data":"71eaf37429160bf74cfc5607a0e4c329f259ae37a19149bd13e67c10e4c6e03c"} Jul 02 19:06:47.227078 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:47.227053 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-hrhd2" event={"ID":"e09cfa31-6ce1-485a-9539-1817701d1e95","Type":"ContainerStarted","Data":"5ba9da02977e7d49d0051bd3f85476a7e664107e052a642d049d57e105d1b984"} Jul 02 19:06:47.266819 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:47.266764 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.934814235 podStartE2EDuration="6.266747943s" totalImagesPullingTime="3.317599939s" totalInitContainerRuntime="14.333769ms" isStatefulPod=false podCreationTimestamp="2026-07-02 19:06:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:47.265786317 +0000 UTC m=+87.192795032" watchObservedRunningTime="2026-07-02 19:06:47.266747943 +0000 UTC m=+87.193756656" Jul 02 19:06:47.373632 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:47.373527 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:47.442417 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:47.442384 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:47.523954 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:47.523922 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:48.232610 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:48.232564 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-hrhd2" event={"ID":"e09cfa31-6ce1-485a-9539-1817701d1e95","Type":"ContainerStarted","Data":"fe1a31a6f6c5cbfb4a5bcf27334244d50a3bb4c309c3867cca06c044e6dbf0a0"} Jul 02 19:06:48.233065 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:48.232898 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-hrhd2" Jul 02 19:06:48.237935 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:48.237913 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-hrhd2" Jul 02 19:06:48.255151 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:48.255111 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-hrhd2" podStartSLOduration=1.82989093 podStartE2EDuration="3.255097446s" totalImagesPullingTime="1.425206516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:06:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:48.254172634 +0000 UTC m=+88.181181347" watchObservedRunningTime="2026-07-02 19:06:48.255097446 +0000 UTC m=+88.182106157" Jul 02 19:06:48.909411 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:48.909374 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:48.909587 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:48.909463 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:48.914243 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:48.914214 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:49.239653 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:49.239615 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:06:49.761185 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:49.761134 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:49.861019 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:49.860987 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:51.190151 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:51.190110 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:51.246609 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:51.246578 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:52.174036 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:52.174004 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:52.237154 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:52.237122 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:52.810275 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:52.810240 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86c9f6454-b5php"] Jul 02 19:06:52.845504 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:52.845467 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86c78c9564-lzlgp"] Jul 02 19:06:52.849721 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:52.849696 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:52.862707 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:52.862667 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86c78c9564-lzlgp"] Jul 02 19:06:53.008270 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.008233 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-service-ca\") pod \"console-86c78c9564-lzlgp\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.008270 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.008278 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-console-config\") pod \"console-86c78c9564-lzlgp\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.008531 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.008301 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/691855e5-876a-4edd-8ac5-52698dfd24f8-console-oauth-config\") pod \"console-86c78c9564-lzlgp\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.008531 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.008323 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-oauth-serving-cert\") pod \"console-86c78c9564-lzlgp\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.008531 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.008392 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz8s5\" (UniqueName: \"kubernetes.io/projected/691855e5-876a-4edd-8ac5-52698dfd24f8-kube-api-access-pz8s5\") pod \"console-86c78c9564-lzlgp\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.008531 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.008454 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/691855e5-876a-4edd-8ac5-52698dfd24f8-console-serving-cert\") pod \"console-86c78c9564-lzlgp\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.008531 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.008475 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-trusted-ca-bundle\") pod \"console-86c78c9564-lzlgp\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.109529 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.109442 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-console-config\") pod \"console-86c78c9564-lzlgp\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.109529 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.109496 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/691855e5-876a-4edd-8ac5-52698dfd24f8-console-oauth-config\") pod \"console-86c78c9564-lzlgp\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.109720 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.109615 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-oauth-serving-cert\") pod \"console-86c78c9564-lzlgp\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.109720 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.109660 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz8s5\" (UniqueName: \"kubernetes.io/projected/691855e5-876a-4edd-8ac5-52698dfd24f8-kube-api-access-pz8s5\") pod \"console-86c78c9564-lzlgp\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.109837 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.109740 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/691855e5-876a-4edd-8ac5-52698dfd24f8-console-serving-cert\") pod \"console-86c78c9564-lzlgp\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.109837 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.109766 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-trusted-ca-bundle\") pod \"console-86c78c9564-lzlgp\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.109932 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.109846 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-service-ca\") pod \"console-86c78c9564-lzlgp\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.110373 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.110344 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-console-config\") pod \"console-86c78c9564-lzlgp\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.110492 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.110420 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-oauth-serving-cert\") pod \"console-86c78c9564-lzlgp\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.110492 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.110445 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-service-ca\") pod \"console-86c78c9564-lzlgp\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.110604 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.110591 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-trusted-ca-bundle\") pod \"console-86c78c9564-lzlgp\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.112079 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.112056 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/691855e5-876a-4edd-8ac5-52698dfd24f8-console-oauth-config\") pod \"console-86c78c9564-lzlgp\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.112607 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.112588 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/691855e5-876a-4edd-8ac5-52698dfd24f8-console-serving-cert\") pod \"console-86c78c9564-lzlgp\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.118824 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.118805 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz8s5\" (UniqueName: \"kubernetes.io/projected/691855e5-876a-4edd-8ac5-52698dfd24f8-kube-api-access-pz8s5\") pod \"console-86c78c9564-lzlgp\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.158928 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.158899 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:06:53.288503 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:53.288480 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86c78c9564-lzlgp"] Jul 02 19:06:53.291015 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:06:53.290987 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod691855e5_876a_4edd_8ac5_52698dfd24f8.slice/crio-e32b0345c163c74237cdd1ffc48b07272c755aa8b24499e09bf1d8e88485d701 WatchSource:0}: Error finding container e32b0345c163c74237cdd1ffc48b07272c755aa8b24499e09bf1d8e88485d701: Status 404 returned error can't find the container with id e32b0345c163c74237cdd1ffc48b07272c755aa8b24499e09bf1d8e88485d701 Jul 02 19:06:54.257800 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:54.257764 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c78c9564-lzlgp" event={"ID":"691855e5-876a-4edd-8ac5-52698dfd24f8","Type":"ContainerStarted","Data":"3dc7f05da1b93cc02b9e95bda687a582c76fa4a4a8e9d6b85b8cbde0e2c16f69"} Jul 02 19:06:54.257943 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:54.257817 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c78c9564-lzlgp" event={"ID":"691855e5-876a-4edd-8ac5-52698dfd24f8","Type":"ContainerStarted","Data":"e32b0345c163c74237cdd1ffc48b07272c755aa8b24499e09bf1d8e88485d701"} Jul 02 19:06:54.273493 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:54.273443 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86c78c9564-lzlgp" podStartSLOduration=2.273427724 podStartE2EDuration="2.273427724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:06:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:06:54.272663416 +0000 UTC m=+94.199672128" watchObservedRunningTime="2026-07-02 19:06:54.273427724 +0000 UTC m=+94.200436435" Jul 02 19:06:55.119023 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:55.118985 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:06:56.119871 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:56.119834 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:56.181876 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:56.181840 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:58.792082 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:58.792040 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:58.846171 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:58.846140 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:59.390118 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:59.390032 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:59.490436 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:59.490404 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:59.794589 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:59.794552 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:06:59.900871 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:06:59.900810 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:07:01.265433 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:01.265367 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:07:01.366332 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:01.366296 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:07:03.159997 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:03.159961 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:07:03.160466 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:03.160023 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:07:03.164738 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:03.164716 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:07:03.297949 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:03.297923 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:07:03.351742 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:03.351704 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57d75f5485-sg5kb"] Jul 02 19:07:03.466952 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:03.466856 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:07:03.523851 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:03.523817 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:07:04.026539 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:04.026496 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:07:04.097604 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:04.097570 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:07:05.199601 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:05.199567 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:07:05.297111 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:05.297080 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:07:07.164037 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:07.164008 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hssx4" Jul 02 19:07:14.392733 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:14.392693 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:07:14.470735 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:14.470701 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:07:17.830265 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:17.830223 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-86c9f6454-b5php" podUID="3febe46a-7653-486b-a07f-65bb190912c6" containerName="console" containerID="cri-o://f76d1a94a966a2389583b3e673a9ded639c58cbf22cff231cff8d679122d2d9c" gracePeriod=15 Jul 02 19:07:18.059451 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.059428 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:07:18.111132 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.111059 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3febe46a-7653-486b-a07f-65bb190912c6-console-oauth-config\") pod \"3febe46a-7653-486b-a07f-65bb190912c6\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " Jul 02 19:07:18.111132 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.111100 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-service-ca\") pod \"3febe46a-7653-486b-a07f-65bb190912c6\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " Jul 02 19:07:18.111132 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.111114 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-trusted-ca-bundle\") pod \"3febe46a-7653-486b-a07f-65bb190912c6\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " Jul 02 19:07:18.111366 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.111135 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-oauth-serving-cert\") pod \"3febe46a-7653-486b-a07f-65bb190912c6\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " Jul 02 19:07:18.111366 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.111154 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-console-config\") pod \"3febe46a-7653-486b-a07f-65bb190912c6\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " Jul 02 19:07:18.111366 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.111223 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6wpm\" (UniqueName: \"kubernetes.io/projected/3febe46a-7653-486b-a07f-65bb190912c6-kube-api-access-r6wpm\") pod \"3febe46a-7653-486b-a07f-65bb190912c6\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " Jul 02 19:07:18.111366 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.111276 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3febe46a-7653-486b-a07f-65bb190912c6-console-serving-cert\") pod \"3febe46a-7653-486b-a07f-65bb190912c6\" (UID: \"3febe46a-7653-486b-a07f-65bb190912c6\") " Jul 02 19:07:18.111580 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.111552 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-service-ca" (OuterVolumeSpecName: "service-ca") pod "3febe46a-7653-486b-a07f-65bb190912c6" (UID: "3febe46a-7653-486b-a07f-65bb190912c6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:07:18.111646 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.111573 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3febe46a-7653-486b-a07f-65bb190912c6" (UID: "3febe46a-7653-486b-a07f-65bb190912c6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:07:18.111646 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.111581 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3febe46a-7653-486b-a07f-65bb190912c6" (UID: "3febe46a-7653-486b-a07f-65bb190912c6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:07:18.111646 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.111589 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-console-config" (OuterVolumeSpecName: "console-config") pod "3febe46a-7653-486b-a07f-65bb190912c6" (UID: "3febe46a-7653-486b-a07f-65bb190912c6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:07:18.111823 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.111781 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-service-ca\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:07:18.111823 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.111794 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-trusted-ca-bundle\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:07:18.111823 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.111806 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-oauth-serving-cert\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:07:18.111823 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.111816 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3febe46a-7653-486b-a07f-65bb190912c6-console-config\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:07:18.113301 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.113275 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3febe46a-7653-486b-a07f-65bb190912c6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3febe46a-7653-486b-a07f-65bb190912c6" (UID: "3febe46a-7653-486b-a07f-65bb190912c6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:07:18.113423 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.113405 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3febe46a-7653-486b-a07f-65bb190912c6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3febe46a-7653-486b-a07f-65bb190912c6" (UID: "3febe46a-7653-486b-a07f-65bb190912c6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:07:18.113474 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.113423 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3febe46a-7653-486b-a07f-65bb190912c6-kube-api-access-r6wpm" (OuterVolumeSpecName: "kube-api-access-r6wpm") pod "3febe46a-7653-486b-a07f-65bb190912c6" (UID: "3febe46a-7653-486b-a07f-65bb190912c6"). InnerVolumeSpecName "kube-api-access-r6wpm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:07:18.213136 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.213084 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6wpm\" (UniqueName: \"kubernetes.io/projected/3febe46a-7653-486b-a07f-65bb190912c6-kube-api-access-r6wpm\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:07:18.213136 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.213130 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3febe46a-7653-486b-a07f-65bb190912c6-console-serving-cert\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:07:18.213136 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.213140 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3febe46a-7653-486b-a07f-65bb190912c6-console-oauth-config\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:07:18.354875 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.354832 2566 generic.go:356] "Generic (PLEG): container finished" podID="3febe46a-7653-486b-a07f-65bb190912c6" containerID="f76d1a94a966a2389583b3e673a9ded639c58cbf22cff231cff8d679122d2d9c" exitCode=0 Jul 02 19:07:18.355040 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.354882 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c9f6454-b5php" Jul 02 19:07:18.355040 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.354880 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c9f6454-b5php" event={"ID":"3febe46a-7653-486b-a07f-65bb190912c6","Type":"ContainerDied","Data":"f76d1a94a966a2389583b3e673a9ded639c58cbf22cff231cff8d679122d2d9c"} Jul 02 19:07:18.355040 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.355017 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c9f6454-b5php" event={"ID":"3febe46a-7653-486b-a07f-65bb190912c6","Type":"ContainerDied","Data":"9b92f97f2040ce710657d980b2ffd54bd60daffdb3fdc11fdf184b6c326c9cc0"} Jul 02 19:07:18.355142 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.355044 2566 scope.go:117] "RemoveContainer" containerID="f76d1a94a966a2389583b3e673a9ded639c58cbf22cff231cff8d679122d2d9c" Jul 02 19:07:18.363343 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.363324 2566 scope.go:117] "RemoveContainer" containerID="f76d1a94a966a2389583b3e673a9ded639c58cbf22cff231cff8d679122d2d9c" Jul 02 19:07:18.363592 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:07:18.363572 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f76d1a94a966a2389583b3e673a9ded639c58cbf22cff231cff8d679122d2d9c\": container with ID starting with f76d1a94a966a2389583b3e673a9ded639c58cbf22cff231cff8d679122d2d9c not found: ID does not exist" containerID="f76d1a94a966a2389583b3e673a9ded639c58cbf22cff231cff8d679122d2d9c" Jul 02 19:07:18.363638 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.363600 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f76d1a94a966a2389583b3e673a9ded639c58cbf22cff231cff8d679122d2d9c"} err="failed to get container status \"f76d1a94a966a2389583b3e673a9ded639c58cbf22cff231cff8d679122d2d9c\": rpc error: code = NotFound desc = could not find container \"f76d1a94a966a2389583b3e673a9ded639c58cbf22cff231cff8d679122d2d9c\": container with ID starting with f76d1a94a966a2389583b3e673a9ded639c58cbf22cff231cff8d679122d2d9c not found: ID does not exist" Jul 02 19:07:18.375006 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.374979 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86c9f6454-b5php"] Jul 02 19:07:18.378332 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.378311 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86c9f6454-b5php"] Jul 02 19:07:18.843418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:18.843381 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3febe46a-7653-486b-a07f-65bb190912c6" path="/var/lib/kubelet/pods/3febe46a-7653-486b-a07f-65bb190912c6/volumes" Jul 02 19:07:28.371285 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:28.371226 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-57d75f5485-sg5kb" podUID="c55205c4-e3ed-44ac-89d7-12c11cb1913b" containerName="console" containerID="cri-o://720f95ad6087a33935ace5e56184af0bb6f6f0738d9fffb9cf46c0aa44f2d273" gracePeriod=15 Jul 02 19:07:28.610040 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:28.610016 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:07:28.693136 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:28.693053 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c55205c4-e3ed-44ac-89d7-12c11cb1913b-console-config\") pod \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " Jul 02 19:07:28.693136 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:28.693093 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c55205c4-e3ed-44ac-89d7-12c11cb1913b-oauth-serving-cert\") pod \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " Jul 02 19:07:28.693359 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:28.693140 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c55205c4-e3ed-44ac-89d7-12c11cb1913b-console-serving-cert\") pod \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " Jul 02 19:07:28.693359 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:28.693177 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9j82g\" (UniqueName: \"kubernetes.io/projected/c55205c4-e3ed-44ac-89d7-12c11cb1913b-kube-api-access-9j82g\") pod \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " Jul 02 19:07:28.693359 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:28.693309 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c55205c4-e3ed-44ac-89d7-12c11cb1913b-console-oauth-config\") pod \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " Jul 02 19:07:28.693508 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:28.693366 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c55205c4-e3ed-44ac-89d7-12c11cb1913b-service-ca\") pod \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\" (UID: \"c55205c4-e3ed-44ac-89d7-12c11cb1913b\") " Jul 02 19:07:28.693555 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:28.693520 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c55205c4-e3ed-44ac-89d7-12c11cb1913b-console-config" (OuterVolumeSpecName: "console-config") pod "c55205c4-e3ed-44ac-89d7-12c11cb1913b" (UID: "c55205c4-e3ed-44ac-89d7-12c11cb1913b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:07:28.693555 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:28.693528 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c55205c4-e3ed-44ac-89d7-12c11cb1913b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c55205c4-e3ed-44ac-89d7-12c11cb1913b" (UID: "c55205c4-e3ed-44ac-89d7-12c11cb1913b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:07:28.693763 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:28.693743 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c55205c4-e3ed-44ac-89d7-12c11cb1913b-console-config\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:07:28.693835 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:28.693769 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c55205c4-e3ed-44ac-89d7-12c11cb1913b-oauth-serving-cert\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:07:28.693835 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:28.693779 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c55205c4-e3ed-44ac-89d7-12c11cb1913b-service-ca" (OuterVolumeSpecName: "service-ca") pod "c55205c4-e3ed-44ac-89d7-12c11cb1913b" (UID: "c55205c4-e3ed-44ac-89d7-12c11cb1913b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:07:28.695324 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:28.695304 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c55205c4-e3ed-44ac-89d7-12c11cb1913b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c55205c4-e3ed-44ac-89d7-12c11cb1913b" (UID: "c55205c4-e3ed-44ac-89d7-12c11cb1913b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:07:28.695399 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:28.695332 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c55205c4-e3ed-44ac-89d7-12c11cb1913b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c55205c4-e3ed-44ac-89d7-12c11cb1913b" (UID: "c55205c4-e3ed-44ac-89d7-12c11cb1913b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:07:28.695399 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:28.695382 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c55205c4-e3ed-44ac-89d7-12c11cb1913b-kube-api-access-9j82g" (OuterVolumeSpecName: "kube-api-access-9j82g") pod "c55205c4-e3ed-44ac-89d7-12c11cb1913b" (UID: "c55205c4-e3ed-44ac-89d7-12c11cb1913b"). InnerVolumeSpecName "kube-api-access-9j82g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:07:28.794909 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:28.794874 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c55205c4-e3ed-44ac-89d7-12c11cb1913b-console-serving-cert\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:07:28.794909 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:28.794905 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9j82g\" (UniqueName: \"kubernetes.io/projected/c55205c4-e3ed-44ac-89d7-12c11cb1913b-kube-api-access-9j82g\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:07:28.794909 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:28.794918 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c55205c4-e3ed-44ac-89d7-12c11cb1913b-console-oauth-config\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:07:28.795128 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:28.794932 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c55205c4-e3ed-44ac-89d7-12c11cb1913b-service-ca\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:07:29.396754 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:29.396717 2566 generic.go:356] "Generic (PLEG): container finished" podID="c55205c4-e3ed-44ac-89d7-12c11cb1913b" containerID="720f95ad6087a33935ace5e56184af0bb6f6f0738d9fffb9cf46c0aa44f2d273" exitCode=0 Jul 02 19:07:29.397202 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:29.396761 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d75f5485-sg5kb" Jul 02 19:07:29.397202 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:29.396808 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d75f5485-sg5kb" event={"ID":"c55205c4-e3ed-44ac-89d7-12c11cb1913b","Type":"ContainerDied","Data":"720f95ad6087a33935ace5e56184af0bb6f6f0738d9fffb9cf46c0aa44f2d273"} Jul 02 19:07:29.397202 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:29.396857 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d75f5485-sg5kb" event={"ID":"c55205c4-e3ed-44ac-89d7-12c11cb1913b","Type":"ContainerDied","Data":"975bb9096a11d18d37267e8d9ed67a8fba2e7de92f7c046186fd7480996c98c2"} Jul 02 19:07:29.397202 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:29.396879 2566 scope.go:117] "RemoveContainer" containerID="720f95ad6087a33935ace5e56184af0bb6f6f0738d9fffb9cf46c0aa44f2d273" Jul 02 19:07:29.405000 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:29.404984 2566 scope.go:117] "RemoveContainer" containerID="720f95ad6087a33935ace5e56184af0bb6f6f0738d9fffb9cf46c0aa44f2d273" Jul 02 19:07:29.405260 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:07:29.405242 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"720f95ad6087a33935ace5e56184af0bb6f6f0738d9fffb9cf46c0aa44f2d273\": container with ID starting with 720f95ad6087a33935ace5e56184af0bb6f6f0738d9fffb9cf46c0aa44f2d273 not found: ID does not exist" containerID="720f95ad6087a33935ace5e56184af0bb6f6f0738d9fffb9cf46c0aa44f2d273" Jul 02 19:07:29.405322 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:29.405271 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"720f95ad6087a33935ace5e56184af0bb6f6f0738d9fffb9cf46c0aa44f2d273"} err="failed to get container status \"720f95ad6087a33935ace5e56184af0bb6f6f0738d9fffb9cf46c0aa44f2d273\": rpc error: code = NotFound desc = could not find container \"720f95ad6087a33935ace5e56184af0bb6f6f0738d9fffb9cf46c0aa44f2d273\": container with ID starting with 720f95ad6087a33935ace5e56184af0bb6f6f0738d9fffb9cf46c0aa44f2d273 not found: ID does not exist" Jul 02 19:07:29.412271 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:29.412248 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57d75f5485-sg5kb"] Jul 02 19:07:29.415901 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:29.415881 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57d75f5485-sg5kb"] Jul 02 19:07:30.843803 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:30.843773 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c55205c4-e3ed-44ac-89d7-12c11cb1913b" path="/var/lib/kubelet/pods/c55205c4-e3ed-44ac-89d7-12c11cb1913b/volumes" Jul 02 19:07:59.872059 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:07:59.872025 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:00.218919 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:00.218835 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:00.943746 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:00.943706 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 19:08:00.944211 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:00.944080 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="alertmanager" containerID="cri-o://9ec02cecdc81bc54ad3d0caa7b4baa1c66b2a1f9dd30ea6f09c64feb96263931" gracePeriod=120 Jul 02 19:08:00.944211 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:00.944130 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="kube-rbac-proxy-metric" containerID="cri-o://61fbcb9b07310d85695d98ac08c334ac96a1819791bbe8ebf2787ad02a64ed99" gracePeriod=120 Jul 02 19:08:00.944211 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:00.944152 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="kube-rbac-proxy-web" containerID="cri-o://d783be1bcfaa3595f2185212a7057938130e7814baa580a089aa2d8bbda760c2" gracePeriod=120 Jul 02 19:08:00.944211 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:00.944167 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="prom-label-proxy" containerID="cri-o://71eaf37429160bf74cfc5607a0e4c329f259ae37a19149bd13e67c10e4c6e03c" gracePeriod=120 Jul 02 19:08:00.944416 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:00.944192 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="kube-rbac-proxy" containerID="cri-o://a97c6b86814e47a3132c58b1c167a598550e9f47184893552b9c42cdadea50cd" gracePeriod=120 Jul 02 19:08:00.944416 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:00.944208 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="config-reloader" containerID="cri-o://a5f1074c73fba7ab554bc11188b7b04d14d3d17d9268fba500058ad3135cfe8b" gracePeriod=120 Jul 02 19:08:01.517082 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:01.517050 2566 generic.go:356] "Generic (PLEG): container finished" podID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerID="71eaf37429160bf74cfc5607a0e4c329f259ae37a19149bd13e67c10e4c6e03c" exitCode=0 Jul 02 19:08:01.517082 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:01.517074 2566 generic.go:356] "Generic (PLEG): container finished" podID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerID="61fbcb9b07310d85695d98ac08c334ac96a1819791bbe8ebf2787ad02a64ed99" exitCode=0 Jul 02 19:08:01.517082 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:01.517081 2566 generic.go:356] "Generic (PLEG): container finished" podID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerID="a97c6b86814e47a3132c58b1c167a598550e9f47184893552b9c42cdadea50cd" exitCode=0 Jul 02 19:08:01.517082 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:01.517088 2566 generic.go:356] "Generic (PLEG): container finished" podID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerID="a5f1074c73fba7ab554bc11188b7b04d14d3d17d9268fba500058ad3135cfe8b" exitCode=0 Jul 02 19:08:01.517082 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:01.517093 2566 generic.go:356] "Generic (PLEG): container finished" podID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerID="9ec02cecdc81bc54ad3d0caa7b4baa1c66b2a1f9dd30ea6f09c64feb96263931" exitCode=0 Jul 02 19:08:01.517445 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:01.517127 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5821ec4-122b-4c9b-b413-83c3e7b026c4","Type":"ContainerDied","Data":"71eaf37429160bf74cfc5607a0e4c329f259ae37a19149bd13e67c10e4c6e03c"} Jul 02 19:08:01.517445 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:01.517204 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5821ec4-122b-4c9b-b413-83c3e7b026c4","Type":"ContainerDied","Data":"61fbcb9b07310d85695d98ac08c334ac96a1819791bbe8ebf2787ad02a64ed99"} Jul 02 19:08:01.517445 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:01.517244 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5821ec4-122b-4c9b-b413-83c3e7b026c4","Type":"ContainerDied","Data":"a97c6b86814e47a3132c58b1c167a598550e9f47184893552b9c42cdadea50cd"} Jul 02 19:08:01.517445 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:01.517280 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5821ec4-122b-4c9b-b413-83c3e7b026c4","Type":"ContainerDied","Data":"a5f1074c73fba7ab554bc11188b7b04d14d3d17d9268fba500058ad3135cfe8b"} Jul 02 19:08:01.517445 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:01.517320 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5821ec4-122b-4c9b-b413-83c3e7b026c4","Type":"ContainerDied","Data":"9ec02cecdc81bc54ad3d0caa7b4baa1c66b2a1f9dd30ea6f09c64feb96263931"} Jul 02 19:08:02.185390 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.185368 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.350100 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.350004 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-web-config\") pod \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " Jul 02 19:08:02.350100 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.350061 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " Jul 02 19:08:02.350100 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.350092 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-config-volume\") pod \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " Jul 02 19:08:02.350384 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.350123 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e5821ec4-122b-4c9b-b413-83c3e7b026c4-alertmanager-main-db\") pod \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " Jul 02 19:08:02.350384 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.350167 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e5821ec4-122b-4c9b-b413-83c3e7b026c4-metrics-client-ca\") pod \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " Jul 02 19:08:02.350384 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.350193 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-kube-rbac-proxy\") pod \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " Jul 02 19:08:02.350384 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.350222 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e5821ec4-122b-4c9b-b413-83c3e7b026c4-tls-assets\") pod \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " Jul 02 19:08:02.350384 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.350248 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-kube-rbac-proxy-web\") pod \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " Jul 02 19:08:02.350384 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.350286 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m78fq\" (UniqueName: \"kubernetes.io/projected/e5821ec4-122b-4c9b-b413-83c3e7b026c4-kube-api-access-m78fq\") pod \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " Jul 02 19:08:02.350384 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.350317 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-main-tls\") pod \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " Jul 02 19:08:02.350384 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.350340 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-cluster-tls-config\") pod \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " Jul 02 19:08:02.350384 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.350368 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5821ec4-122b-4c9b-b413-83c3e7b026c4-alertmanager-trusted-ca-bundle\") pod \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " Jul 02 19:08:02.350851 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.350393 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e5821ec4-122b-4c9b-b413-83c3e7b026c4-config-out\") pod \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\" (UID: \"e5821ec4-122b-4c9b-b413-83c3e7b026c4\") " Jul 02 19:08:02.350851 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.350517 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5821ec4-122b-4c9b-b413-83c3e7b026c4-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "e5821ec4-122b-4c9b-b413-83c3e7b026c4" (UID: "e5821ec4-122b-4c9b-b413-83c3e7b026c4"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:08:02.350851 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.350571 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5821ec4-122b-4c9b-b413-83c3e7b026c4-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "e5821ec4-122b-4c9b-b413-83c3e7b026c4" (UID: "e5821ec4-122b-4c9b-b413-83c3e7b026c4"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:08:02.350851 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.350654 2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e5821ec4-122b-4c9b-b413-83c3e7b026c4-alertmanager-main-db\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:08:02.350851 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.350698 2566 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e5821ec4-122b-4c9b-b413-83c3e7b026c4-metrics-client-ca\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:08:02.353058 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.352975 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "e5821ec4-122b-4c9b-b413-83c3e7b026c4" (UID: "e5821ec4-122b-4c9b-b413-83c3e7b026c4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:08:02.353058 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.352976 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5821ec4-122b-4c9b-b413-83c3e7b026c4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e5821ec4-122b-4c9b-b413-83c3e7b026c4" (UID: "e5821ec4-122b-4c9b-b413-83c3e7b026c4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:08:02.353058 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.353020 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5821ec4-122b-4c9b-b413-83c3e7b026c4-kube-api-access-m78fq" (OuterVolumeSpecName: "kube-api-access-m78fq") pod "e5821ec4-122b-4c9b-b413-83c3e7b026c4" (UID: "e5821ec4-122b-4c9b-b413-83c3e7b026c4"). InnerVolumeSpecName "kube-api-access-m78fq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:08:02.353455 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.353400 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5821ec4-122b-4c9b-b413-83c3e7b026c4-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "e5821ec4-122b-4c9b-b413-83c3e7b026c4" (UID: "e5821ec4-122b-4c9b-b413-83c3e7b026c4"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:08:02.353455 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.353403 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "e5821ec4-122b-4c9b-b413-83c3e7b026c4" (UID: "e5821ec4-122b-4c9b-b413-83c3e7b026c4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:08:02.353455 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.353425 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-config-volume" (OuterVolumeSpecName: "config-volume") pod "e5821ec4-122b-4c9b-b413-83c3e7b026c4" (UID: "e5821ec4-122b-4c9b-b413-83c3e7b026c4"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:08:02.354350 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.354323 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5821ec4-122b-4c9b-b413-83c3e7b026c4-config-out" (OuterVolumeSpecName: "config-out") pod "e5821ec4-122b-4c9b-b413-83c3e7b026c4" (UID: "e5821ec4-122b-4c9b-b413-83c3e7b026c4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:08:02.354974 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.354948 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "e5821ec4-122b-4c9b-b413-83c3e7b026c4" (UID: "e5821ec4-122b-4c9b-b413-83c3e7b026c4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:08:02.355076 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.355055 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "e5821ec4-122b-4c9b-b413-83c3e7b026c4" (UID: "e5821ec4-122b-4c9b-b413-83c3e7b026c4"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:08:02.358332 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.358308 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "e5821ec4-122b-4c9b-b413-83c3e7b026c4" (UID: "e5821ec4-122b-4c9b-b413-83c3e7b026c4"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:08:02.364590 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.364570 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-web-config" (OuterVolumeSpecName: "web-config") pod "e5821ec4-122b-4c9b-b413-83c3e7b026c4" (UID: "e5821ec4-122b-4c9b-b413-83c3e7b026c4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:08:02.451224 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.451188 2566 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-web-config\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:08:02.451224 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.451220 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:08:02.451224 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.451229 2566 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-config-volume\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:08:02.451455 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.451239 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:08:02.451455 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.451248 2566 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e5821ec4-122b-4c9b-b413-83c3e7b026c4-tls-assets\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:08:02.451455 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.451256 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:08:02.451455 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.451265 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m78fq\" (UniqueName: \"kubernetes.io/projected/e5821ec4-122b-4c9b-b413-83c3e7b026c4-kube-api-access-m78fq\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:08:02.451455 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.451274 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-secret-alertmanager-main-tls\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:08:02.451455 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.451283 2566 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e5821ec4-122b-4c9b-b413-83c3e7b026c4-cluster-tls-config\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:08:02.451455 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.451291 2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e5821ec4-122b-4c9b-b413-83c3e7b026c4-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:08:02.451455 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.451299 2566 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e5821ec4-122b-4c9b-b413-83c3e7b026c4-config-out\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:08:02.523893 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.523861 2566 generic.go:356] "Generic (PLEG): container finished" podID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerID="d783be1bcfaa3595f2185212a7057938130e7814baa580a089aa2d8bbda760c2" exitCode=0 Jul 02 19:08:02.524062 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.523943 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5821ec4-122b-4c9b-b413-83c3e7b026c4","Type":"ContainerDied","Data":"d783be1bcfaa3595f2185212a7057938130e7814baa580a089aa2d8bbda760c2"} Jul 02 19:08:02.524062 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.523973 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.524062 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.524040 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e5821ec4-122b-4c9b-b413-83c3e7b026c4","Type":"ContainerDied","Data":"329998fd0fe9e574a8e873603c097ae7ed4cde4554e3565812f718763c2f5197"} Jul 02 19:08:02.524227 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.524073 2566 scope.go:117] "RemoveContainer" containerID="71eaf37429160bf74cfc5607a0e4c329f259ae37a19149bd13e67c10e4c6e03c" Jul 02 19:08:02.532548 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.532529 2566 scope.go:117] "RemoveContainer" containerID="61fbcb9b07310d85695d98ac08c334ac96a1819791bbe8ebf2787ad02a64ed99" Jul 02 19:08:02.539367 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.539348 2566 scope.go:117] "RemoveContainer" containerID="a97c6b86814e47a3132c58b1c167a598550e9f47184893552b9c42cdadea50cd" Jul 02 19:08:02.545626 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.545606 2566 scope.go:117] "RemoveContainer" containerID="d783be1bcfaa3595f2185212a7057938130e7814baa580a089aa2d8bbda760c2" Jul 02 19:08:02.548456 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.548436 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 19:08:02.553130 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.553113 2566 scope.go:117] "RemoveContainer" containerID="a5f1074c73fba7ab554bc11188b7b04d14d3d17d9268fba500058ad3135cfe8b" Jul 02 19:08:02.553224 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.553205 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 19:08:02.559667 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.559651 2566 scope.go:117] "RemoveContainer" containerID="9ec02cecdc81bc54ad3d0caa7b4baa1c66b2a1f9dd30ea6f09c64feb96263931" Jul 02 19:08:02.566179 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.566163 2566 scope.go:117] "RemoveContainer" containerID="cd1cf0e8c17d064eec3bf26389d8f0189db8e5488f5130dee82977c982579ec4" Jul 02 19:08:02.573763 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.573747 2566 scope.go:117] "RemoveContainer" containerID="71eaf37429160bf74cfc5607a0e4c329f259ae37a19149bd13e67c10e4c6e03c" Jul 02 19:08:02.574011 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:08:02.573995 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71eaf37429160bf74cfc5607a0e4c329f259ae37a19149bd13e67c10e4c6e03c\": container with ID starting with 71eaf37429160bf74cfc5607a0e4c329f259ae37a19149bd13e67c10e4c6e03c not found: ID does not exist" containerID="71eaf37429160bf74cfc5607a0e4c329f259ae37a19149bd13e67c10e4c6e03c" Jul 02 19:08:02.574068 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.574019 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71eaf37429160bf74cfc5607a0e4c329f259ae37a19149bd13e67c10e4c6e03c"} err="failed to get container status \"71eaf37429160bf74cfc5607a0e4c329f259ae37a19149bd13e67c10e4c6e03c\": rpc error: code = NotFound desc = could not find container \"71eaf37429160bf74cfc5607a0e4c329f259ae37a19149bd13e67c10e4c6e03c\": container with ID starting with 71eaf37429160bf74cfc5607a0e4c329f259ae37a19149bd13e67c10e4c6e03c not found: ID does not exist" Jul 02 19:08:02.574068 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.574042 2566 scope.go:117] "RemoveContainer" containerID="61fbcb9b07310d85695d98ac08c334ac96a1819791bbe8ebf2787ad02a64ed99" Jul 02 19:08:02.574272 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:08:02.574237 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61fbcb9b07310d85695d98ac08c334ac96a1819791bbe8ebf2787ad02a64ed99\": container with ID starting with 61fbcb9b07310d85695d98ac08c334ac96a1819791bbe8ebf2787ad02a64ed99 not found: ID does not exist" containerID="61fbcb9b07310d85695d98ac08c334ac96a1819791bbe8ebf2787ad02a64ed99" Jul 02 19:08:02.574334 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.574281 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61fbcb9b07310d85695d98ac08c334ac96a1819791bbe8ebf2787ad02a64ed99"} err="failed to get container status \"61fbcb9b07310d85695d98ac08c334ac96a1819791bbe8ebf2787ad02a64ed99\": rpc error: code = NotFound desc = could not find container \"61fbcb9b07310d85695d98ac08c334ac96a1819791bbe8ebf2787ad02a64ed99\": container with ID starting with 61fbcb9b07310d85695d98ac08c334ac96a1819791bbe8ebf2787ad02a64ed99 not found: ID does not exist" Jul 02 19:08:02.574334 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.574304 2566 scope.go:117] "RemoveContainer" containerID="a97c6b86814e47a3132c58b1c167a598550e9f47184893552b9c42cdadea50cd" Jul 02 19:08:02.574523 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:08:02.574504 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a97c6b86814e47a3132c58b1c167a598550e9f47184893552b9c42cdadea50cd\": container with ID starting with a97c6b86814e47a3132c58b1c167a598550e9f47184893552b9c42cdadea50cd not found: ID does not exist" containerID="a97c6b86814e47a3132c58b1c167a598550e9f47184893552b9c42cdadea50cd" Jul 02 19:08:02.574560 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.574527 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a97c6b86814e47a3132c58b1c167a598550e9f47184893552b9c42cdadea50cd"} err="failed to get container status \"a97c6b86814e47a3132c58b1c167a598550e9f47184893552b9c42cdadea50cd\": rpc error: code = NotFound desc = could not find container \"a97c6b86814e47a3132c58b1c167a598550e9f47184893552b9c42cdadea50cd\": container with ID starting with a97c6b86814e47a3132c58b1c167a598550e9f47184893552b9c42cdadea50cd not found: ID does not exist" Jul 02 19:08:02.574560 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.574540 2566 scope.go:117] "RemoveContainer" containerID="d783be1bcfaa3595f2185212a7057938130e7814baa580a089aa2d8bbda760c2" Jul 02 19:08:02.574736 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:08:02.574718 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d783be1bcfaa3595f2185212a7057938130e7814baa580a089aa2d8bbda760c2\": container with ID starting with d783be1bcfaa3595f2185212a7057938130e7814baa580a089aa2d8bbda760c2 not found: ID does not exist" containerID="d783be1bcfaa3595f2185212a7057938130e7814baa580a089aa2d8bbda760c2" Jul 02 19:08:02.574790 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.574744 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d783be1bcfaa3595f2185212a7057938130e7814baa580a089aa2d8bbda760c2"} err="failed to get container status \"d783be1bcfaa3595f2185212a7057938130e7814baa580a089aa2d8bbda760c2\": rpc error: code = NotFound desc = could not find container \"d783be1bcfaa3595f2185212a7057938130e7814baa580a089aa2d8bbda760c2\": container with ID starting with d783be1bcfaa3595f2185212a7057938130e7814baa580a089aa2d8bbda760c2 not found: ID does not exist" Jul 02 19:08:02.574790 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.574758 2566 scope.go:117] "RemoveContainer" containerID="a5f1074c73fba7ab554bc11188b7b04d14d3d17d9268fba500058ad3135cfe8b" Jul 02 19:08:02.575055 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:08:02.575037 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5f1074c73fba7ab554bc11188b7b04d14d3d17d9268fba500058ad3135cfe8b\": container with ID starting with a5f1074c73fba7ab554bc11188b7b04d14d3d17d9268fba500058ad3135cfe8b not found: ID does not exist" containerID="a5f1074c73fba7ab554bc11188b7b04d14d3d17d9268fba500058ad3135cfe8b" Jul 02 19:08:02.575125 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.575062 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5f1074c73fba7ab554bc11188b7b04d14d3d17d9268fba500058ad3135cfe8b"} err="failed to get container status \"a5f1074c73fba7ab554bc11188b7b04d14d3d17d9268fba500058ad3135cfe8b\": rpc error: code = NotFound desc = could not find container \"a5f1074c73fba7ab554bc11188b7b04d14d3d17d9268fba500058ad3135cfe8b\": container with ID starting with a5f1074c73fba7ab554bc11188b7b04d14d3d17d9268fba500058ad3135cfe8b not found: ID does not exist" Jul 02 19:08:02.575125 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.575081 2566 scope.go:117] "RemoveContainer" containerID="9ec02cecdc81bc54ad3d0caa7b4baa1c66b2a1f9dd30ea6f09c64feb96263931" Jul 02 19:08:02.575333 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:08:02.575318 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ec02cecdc81bc54ad3d0caa7b4baa1c66b2a1f9dd30ea6f09c64feb96263931\": container with ID starting with 9ec02cecdc81bc54ad3d0caa7b4baa1c66b2a1f9dd30ea6f09c64feb96263931 not found: ID does not exist" containerID="9ec02cecdc81bc54ad3d0caa7b4baa1c66b2a1f9dd30ea6f09c64feb96263931" Jul 02 19:08:02.575380 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.575338 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ec02cecdc81bc54ad3d0caa7b4baa1c66b2a1f9dd30ea6f09c64feb96263931"} err="failed to get container status \"9ec02cecdc81bc54ad3d0caa7b4baa1c66b2a1f9dd30ea6f09c64feb96263931\": rpc error: code = NotFound desc = could not find container \"9ec02cecdc81bc54ad3d0caa7b4baa1c66b2a1f9dd30ea6f09c64feb96263931\": container with ID starting with 9ec02cecdc81bc54ad3d0caa7b4baa1c66b2a1f9dd30ea6f09c64feb96263931 not found: ID does not exist" Jul 02 19:08:02.575380 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.575350 2566 scope.go:117] "RemoveContainer" containerID="cd1cf0e8c17d064eec3bf26389d8f0189db8e5488f5130dee82977c982579ec4" Jul 02 19:08:02.575571 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:08:02.575553 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd1cf0e8c17d064eec3bf26389d8f0189db8e5488f5130dee82977c982579ec4\": container with ID starting with cd1cf0e8c17d064eec3bf26389d8f0189db8e5488f5130dee82977c982579ec4 not found: ID does not exist" containerID="cd1cf0e8c17d064eec3bf26389d8f0189db8e5488f5130dee82977c982579ec4" Jul 02 19:08:02.575689 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.575575 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd1cf0e8c17d064eec3bf26389d8f0189db8e5488f5130dee82977c982579ec4"} err="failed to get container status \"cd1cf0e8c17d064eec3bf26389d8f0189db8e5488f5130dee82977c982579ec4\": rpc error: code = NotFound desc = could not find container \"cd1cf0e8c17d064eec3bf26389d8f0189db8e5488f5130dee82977c982579ec4\": container with ID starting with cd1cf0e8c17d064eec3bf26389d8f0189db8e5488f5130dee82977c982579ec4 not found: ID does not exist" Jul 02 19:08:02.577283 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577263 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 19:08:02.577506 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577496 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="kube-rbac-proxy" Jul 02 19:08:02.577569 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577508 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="kube-rbac-proxy" Jul 02 19:08:02.577569 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577519 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="config-reloader" Jul 02 19:08:02.577569 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577525 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="config-reloader" Jul 02 19:08:02.577569 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577533 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="init-config-reloader" Jul 02 19:08:02.577569 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577539 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="init-config-reloader" Jul 02 19:08:02.577569 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577547 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="kube-rbac-proxy-metric" Jul 02 19:08:02.577569 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577552 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="kube-rbac-proxy-metric" Jul 02 19:08:02.577569 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577558 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="prom-label-proxy" Jul 02 19:08:02.577569 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577563 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="prom-label-proxy" Jul 02 19:08:02.577569 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577570 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="alertmanager" Jul 02 19:08:02.577867 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577575 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="alertmanager" Jul 02 19:08:02.577867 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577581 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="kube-rbac-proxy-web" Jul 02 19:08:02.577867 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577586 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="kube-rbac-proxy-web" Jul 02 19:08:02.577867 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577596 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3febe46a-7653-486b-a07f-65bb190912c6" containerName="console" Jul 02 19:08:02.577867 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577601 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3febe46a-7653-486b-a07f-65bb190912c6" containerName="console" Jul 02 19:08:02.577867 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577608 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c55205c4-e3ed-44ac-89d7-12c11cb1913b" containerName="console" Jul 02 19:08:02.577867 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577613 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c55205c4-e3ed-44ac-89d7-12c11cb1913b" containerName="console" Jul 02 19:08:02.577867 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577651 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="prom-label-proxy" Jul 02 19:08:02.577867 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577658 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c55205c4-e3ed-44ac-89d7-12c11cb1913b" containerName="console" Jul 02 19:08:02.577867 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577664 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="kube-rbac-proxy-web" Jul 02 19:08:02.577867 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577670 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3febe46a-7653-486b-a07f-65bb190912c6" containerName="console" Jul 02 19:08:02.577867 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577693 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="alertmanager" Jul 02 19:08:02.577867 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577700 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="kube-rbac-proxy" Jul 02 19:08:02.577867 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577705 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="kube-rbac-proxy-metric" Jul 02 19:08:02.577867 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.577710 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" containerName="config-reloader" Jul 02 19:08:02.580922 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.580907 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.583561 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.583497 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 02 19:08:02.583561 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.583536 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 02 19:08:02.583561 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.583556 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 02 19:08:02.583856 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.583563 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2cmkd\"" Jul 02 19:08:02.583856 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.583605 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 02 19:08:02.583856 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.583617 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 02 19:08:02.583988 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.583928 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 02 19:08:02.584064 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.584036 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 02 19:08:02.584438 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.584422 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 02 19:08:02.588671 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.588650 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 02 19:08:02.598186 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.598164 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 19:08:02.652459 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.652361 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.652459 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.652403 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.652459 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.652441 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8mgr\" (UniqueName: \"kubernetes.io/projected/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-kube-api-access-r8mgr\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.652699 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.652478 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-config-out\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.652699 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.652513 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.652699 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.652546 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-config-volume\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.652699 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.652565 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-web-config\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.652699 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.652582 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.652699 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.652652 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.652880 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.652705 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.652880 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.652740 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.652880 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.652756 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.652880 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.652778 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.753729 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.753667 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.753729 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.753734 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.753969 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.753754 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.753969 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.753778 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8mgr\" (UniqueName: \"kubernetes.io/projected/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-kube-api-access-r8mgr\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.753969 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.753795 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-config-out\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.753969 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.753811 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.753969 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.753837 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-config-volume\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.753969 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.753856 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-web-config\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.753969 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.753917 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.754319 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.753970 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.754319 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.754035 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.754319 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.754067 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.754319 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.754093 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.754319 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.754220 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.754843 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.754819 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.755467 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.755434 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.756787 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.756760 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.756999 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.756934 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-web-config\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.757103 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.757052 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-config-volume\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.757103 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.757072 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.757212 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.757157 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-config-out\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.757265 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.757233 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.757527 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.757511 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.758037 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.758019 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.758777 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.758761 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.762854 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.762835 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8mgr\" (UniqueName: \"kubernetes.io/projected/6cc8242f-eb33-4f83-9cfc-c3b069dc72ac-kube-api-access-r8mgr\") pod \"alertmanager-main-0\" (UID: \"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:02.843161 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.843122 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5821ec4-122b-4c9b-b413-83c3e7b026c4" path="/var/lib/kubelet/pods/e5821ec4-122b-4c9b-b413-83c3e7b026c4/volumes" Jul 02 19:08:02.891065 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:02.891019 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 19:08:03.026194 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:03.026161 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 19:08:03.029041 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:08:03.029014 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cc8242f_eb33_4f83_9cfc_c3b069dc72ac.slice/crio-7b36e9270e937165494e618eca743747f43a5a8184374a20ea2c9f0b414fb3c2 WatchSource:0}: Error finding container 7b36e9270e937165494e618eca743747f43a5a8184374a20ea2c9f0b414fb3c2: Status 404 returned error can't find the container with id 7b36e9270e937165494e618eca743747f43a5a8184374a20ea2c9f0b414fb3c2 Jul 02 19:08:03.529322 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:03.529285 2566 generic.go:356] "Generic (PLEG): container finished" podID="6cc8242f-eb33-4f83-9cfc-c3b069dc72ac" containerID="b442e964294ad4c30ba05c4a6f60d1f9c82c39d8dee07f644c893cc519cbc72a" exitCode=0 Jul 02 19:08:03.529738 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:03.529337 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac","Type":"ContainerDied","Data":"b442e964294ad4c30ba05c4a6f60d1f9c82c39d8dee07f644c893cc519cbc72a"} Jul 02 19:08:03.529738 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:03.529392 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac","Type":"ContainerStarted","Data":"7b36e9270e937165494e618eca743747f43a5a8184374a20ea2c9f0b414fb3c2"} Jul 02 19:08:04.536701 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:04.536645 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac","Type":"ContainerStarted","Data":"df79a1c55bfb18bd7ef2f3fa4175caaf4ff6f9bf110383671b09846ec8c10788"} Jul 02 19:08:04.537167 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:04.536762 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac","Type":"ContainerStarted","Data":"82efaf074249fa277736cef794cb4fd7c2737c25fa2f604bd2fc5226e3dbbcc4"} Jul 02 19:08:04.537167 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:04.536793 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac","Type":"ContainerStarted","Data":"4051b1b3bcdd2cfd884fa8d825c986b315a976658c993bed0217ba087bb94688"} Jul 02 19:08:04.537167 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:04.536818 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac","Type":"ContainerStarted","Data":"127e1b8090ae2cb05ea9819cbfba5e0b7aac399f03859592ce4fc69b741d43f1"} Jul 02 19:08:04.537167 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:04.536846 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac","Type":"ContainerStarted","Data":"fdaab46f27e213205f192d5c6c8c6b763bf5b07f4955ee26854009f5803c783f"} Jul 02 19:08:04.537167 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:04.536869 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6cc8242f-eb33-4f83-9cfc-c3b069dc72ac","Type":"ContainerStarted","Data":"c6c7235e1f76d7061dafd83d1e6326290f101bff91e9b876105512bf1dee2f90"} Jul 02 19:08:04.562226 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:04.562176 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.553224812 podStartE2EDuration="2.562159469s" totalImagesPullingTime="0s" totalInitContainerRuntime="8.934657ms" isStatefulPod=false podCreationTimestamp="2026-07-02 19:08:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:04.560575273 +0000 UTC m=+164.487583988" watchObservedRunningTime="2026-07-02 19:08:04.562159469 +0000 UTC m=+164.489168185" Jul 02 19:08:04.961416 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:04.961332 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt"] Jul 02 19:08:04.964191 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:04.964173 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:04.968475 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:04.968264 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 02 19:08:04.968644 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:04.968622 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 02 19:08:04.968748 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:04.968621 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 02 19:08:04.968816 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:04.968691 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 02 19:08:04.968882 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:04.968866 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-8r74c\"" Jul 02 19:08:04.968942 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:04.968884 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 02 19:08:04.974147 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:04.974125 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 02 19:08:04.979962 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:04.979940 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt"] Jul 02 19:08:05.074871 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.074830 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3ef1c4aa-0869-47bf-b586-659a4cc9b961-federate-client-tls\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.075072 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.074889 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ef1c4aa-0869-47bf-b586-659a4cc9b961-metrics-client-ca\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.075072 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.074929 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3ef1c4aa-0869-47bf-b586-659a4cc9b961-secret-telemeter-client\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.075072 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.074960 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ef1c4aa-0869-47bf-b586-659a4cc9b961-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.075072 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.075002 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3ef1c4aa-0869-47bf-b586-659a4cc9b961-telemeter-client-tls\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.075072 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.075049 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef1c4aa-0869-47bf-b586-659a4cc9b961-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.075323 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.075103 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef1c4aa-0869-47bf-b586-659a4cc9b961-serving-certs-ca-bundle\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.075323 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.075128 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f2l2\" (UniqueName: \"kubernetes.io/projected/3ef1c4aa-0869-47bf-b586-659a4cc9b961-kube-api-access-6f2l2\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.175792 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.175754 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ef1c4aa-0869-47bf-b586-659a4cc9b961-metrics-client-ca\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.176009 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.175817 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3ef1c4aa-0869-47bf-b586-659a4cc9b961-secret-telemeter-client\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.176009 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.175889 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ef1c4aa-0869-47bf-b586-659a4cc9b961-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.176009 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.175952 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3ef1c4aa-0869-47bf-b586-659a4cc9b961-telemeter-client-tls\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.176009 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.175978 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef1c4aa-0869-47bf-b586-659a4cc9b961-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.176219 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.176038 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef1c4aa-0869-47bf-b586-659a4cc9b961-serving-certs-ca-bundle\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.176219 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.176067 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f2l2\" (UniqueName: \"kubernetes.io/projected/3ef1c4aa-0869-47bf-b586-659a4cc9b961-kube-api-access-6f2l2\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.176219 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.176130 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3ef1c4aa-0869-47bf-b586-659a4cc9b961-federate-client-tls\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.176737 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.176708 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ef1c4aa-0869-47bf-b586-659a4cc9b961-metrics-client-ca\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.178331 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.177588 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef1c4aa-0869-47bf-b586-659a4cc9b961-serving-certs-ca-bundle\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.178331 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.177785 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef1c4aa-0869-47bf-b586-659a4cc9b961-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.179825 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.179804 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ef1c4aa-0869-47bf-b586-659a4cc9b961-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.180394 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.180146 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3ef1c4aa-0869-47bf-b586-659a4cc9b961-federate-client-tls\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.180828 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.180801 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3ef1c4aa-0869-47bf-b586-659a4cc9b961-secret-telemeter-client\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.181182 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.181162 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3ef1c4aa-0869-47bf-b586-659a4cc9b961-telemeter-client-tls\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.186245 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.186226 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f2l2\" (UniqueName: \"kubernetes.io/projected/3ef1c4aa-0869-47bf-b586-659a4cc9b961-kube-api-access-6f2l2\") pod \"telemeter-client-6cdbd58d97-mvhrt\" (UID: \"3ef1c4aa-0869-47bf-b586-659a4cc9b961\") " pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.276396 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.276358 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" Jul 02 19:08:05.435766 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.435726 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt"] Jul 02 19:08:05.438510 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:08:05.438485 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ef1c4aa_0869_47bf_b586_659a4cc9b961.slice/crio-e12a28cc4750ebd47a6630af891311bdc08113ead60faf86dc6941ef01db1fbf WatchSource:0}: Error finding container e12a28cc4750ebd47a6630af891311bdc08113ead60faf86dc6941ef01db1fbf: Status 404 returned error can't find the container with id e12a28cc4750ebd47a6630af891311bdc08113ead60faf86dc6941ef01db1fbf Jul 02 19:08:05.542288 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:05.542194 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" event={"ID":"3ef1c4aa-0869-47bf-b586-659a4cc9b961","Type":"ContainerStarted","Data":"e12a28cc4750ebd47a6630af891311bdc08113ead60faf86dc6941ef01db1fbf"} Jul 02 19:08:07.552125 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:07.552093 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" event={"ID":"3ef1c4aa-0869-47bf-b586-659a4cc9b961","Type":"ContainerStarted","Data":"bd8d26fd9aa0fe3c4414c4f93625627bf3d9520cb753aa64f228004fca4e5fcb"} Jul 02 19:08:08.557987 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:08.557950 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" event={"ID":"3ef1c4aa-0869-47bf-b586-659a4cc9b961","Type":"ContainerStarted","Data":"e35317f59064caf19ec350c2c8d1e0792f67321d6b01c8cf77d2d83d6802d33e"} Jul 02 19:08:08.558456 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:08.558007 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" event={"ID":"3ef1c4aa-0869-47bf-b586-659a4cc9b961","Type":"ContainerStarted","Data":"bb5319076046b127e8564ae52d182b8b81cef486e2d035037c471d17bdd7e330"} Jul 02 19:08:08.583216 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:08.583160 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6cdbd58d97-mvhrt" podStartSLOduration=2.574799341 podStartE2EDuration="4.583142403s" totalImagesPullingTime="2.008343062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:08.582093686 +0000 UTC m=+168.509102399" watchObservedRunningTime="2026-07-02 19:08:08.583142403 +0000 UTC m=+168.510151115" Jul 02 19:08:09.334425 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.333265 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b64fc4db7-kvx8b"] Jul 02 19:08:09.337125 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.337104 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.347797 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.347737 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b64fc4db7-kvx8b"] Jul 02 19:08:09.391324 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.391279 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:09.448209 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.448179 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:09.512412 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.512372 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ceec284-1c1e-4882-879d-ca920b9f23f9-console-serving-cert\") pod \"console-6b64fc4db7-kvx8b\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.512563 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.512423 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-oauth-serving-cert\") pod \"console-6b64fc4db7-kvx8b\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.512563 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.512449 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-trusted-ca-bundle\") pod \"console-6b64fc4db7-kvx8b\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.512563 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.512493 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-console-config\") pod \"console-6b64fc4db7-kvx8b\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.512563 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.512538 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdj6q\" (UniqueName: \"kubernetes.io/projected/2ceec284-1c1e-4882-879d-ca920b9f23f9-kube-api-access-mdj6q\") pod \"console-6b64fc4db7-kvx8b\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.512771 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.512587 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-service-ca\") pod \"console-6b64fc4db7-kvx8b\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.512771 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.512625 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ceec284-1c1e-4882-879d-ca920b9f23f9-console-oauth-config\") pod \"console-6b64fc4db7-kvx8b\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.614031 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.613941 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ceec284-1c1e-4882-879d-ca920b9f23f9-console-serving-cert\") pod \"console-6b64fc4db7-kvx8b\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.614031 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.613990 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-oauth-serving-cert\") pod \"console-6b64fc4db7-kvx8b\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.614031 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.614009 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-trusted-ca-bundle\") pod \"console-6b64fc4db7-kvx8b\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.614613 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.614036 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-console-config\") pod \"console-6b64fc4db7-kvx8b\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.614613 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.614058 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdj6q\" (UniqueName: \"kubernetes.io/projected/2ceec284-1c1e-4882-879d-ca920b9f23f9-kube-api-access-mdj6q\") pod \"console-6b64fc4db7-kvx8b\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.614613 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.614086 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-service-ca\") pod \"console-6b64fc4db7-kvx8b\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.614613 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.614249 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ceec284-1c1e-4882-879d-ca920b9f23f9-console-oauth-config\") pod \"console-6b64fc4db7-kvx8b\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.614923 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.614896 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-oauth-serving-cert\") pod \"console-6b64fc4db7-kvx8b\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.614984 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.614896 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-console-config\") pod \"console-6b64fc4db7-kvx8b\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.615130 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.615106 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-trusted-ca-bundle\") pod \"console-6b64fc4db7-kvx8b\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.615168 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.615116 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-service-ca\") pod \"console-6b64fc4db7-kvx8b\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.616477 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.616450 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ceec284-1c1e-4882-879d-ca920b9f23f9-console-serving-cert\") pod \"console-6b64fc4db7-kvx8b\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.616598 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.616581 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ceec284-1c1e-4882-879d-ca920b9f23f9-console-oauth-config\") pod \"console-6b64fc4db7-kvx8b\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.622170 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.622142 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdj6q\" (UniqueName: \"kubernetes.io/projected/2ceec284-1c1e-4882-879d-ca920b9f23f9-kube-api-access-mdj6q\") pod \"console-6b64fc4db7-kvx8b\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.647915 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.647878 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:09.773243 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:09.773205 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b64fc4db7-kvx8b"] Jul 02 19:08:09.776291 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:08:09.776250 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ceec284_1c1e_4882_879d_ca920b9f23f9.slice/crio-30099da7465965235e9d018c0a2deabfe016207e9570c5a32ed12023407bebdb WatchSource:0}: Error finding container 30099da7465965235e9d018c0a2deabfe016207e9570c5a32ed12023407bebdb: Status 404 returned error can't find the container with id 30099da7465965235e9d018c0a2deabfe016207e9570c5a32ed12023407bebdb Jul 02 19:08:10.566465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:10.566428 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b64fc4db7-kvx8b" event={"ID":"2ceec284-1c1e-4882-879d-ca920b9f23f9","Type":"ContainerStarted","Data":"1521acab9bd8bfc1123d9e25c196d67094f9441cf1fdb5b08e621124aa003617"} Jul 02 19:08:10.566650 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:10.566486 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b64fc4db7-kvx8b" event={"ID":"2ceec284-1c1e-4882-879d-ca920b9f23f9","Type":"ContainerStarted","Data":"30099da7465965235e9d018c0a2deabfe016207e9570c5a32ed12023407bebdb"} Jul 02 19:08:10.586099 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:10.586051 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b64fc4db7-kvx8b" podStartSLOduration=1.586035233 podStartE2EDuration="1.586035233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:08:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:08:10.585670677 +0000 UTC m=+170.512679390" watchObservedRunningTime="2026-07-02 19:08:10.586035233 +0000 UTC m=+170.513043946" Jul 02 19:08:19.648359 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:19.648275 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:19.648790 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:19.648597 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:19.653133 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:19.653110 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:20.609497 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:20.609467 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:08:20.651428 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:20.651396 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86c78c9564-lzlgp"] Jul 02 19:08:20.754580 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:20.754548 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:45.676664 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:45.676607 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-86c78c9564-lzlgp" podUID="691855e5-876a-4edd-8ac5-52698dfd24f8" containerName="console" containerID="cri-o://3dc7f05da1b93cc02b9e95bda687a582c76fa4a4a8e9d6b85b8cbde0e2c16f69" gracePeriod=15 Jul 02 19:08:45.915535 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:45.915512 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:08:46.024107 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.024078 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/691855e5-876a-4edd-8ac5-52698dfd24f8-console-serving-cert\") pod \"691855e5-876a-4edd-8ac5-52698dfd24f8\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " Jul 02 19:08:46.024270 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.024118 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/691855e5-876a-4edd-8ac5-52698dfd24f8-console-oauth-config\") pod \"691855e5-876a-4edd-8ac5-52698dfd24f8\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " Jul 02 19:08:46.024270 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.024180 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pz8s5\" (UniqueName: \"kubernetes.io/projected/691855e5-876a-4edd-8ac5-52698dfd24f8-kube-api-access-pz8s5\") pod \"691855e5-876a-4edd-8ac5-52698dfd24f8\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " Jul 02 19:08:46.024270 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.024204 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-trusted-ca-bundle\") pod \"691855e5-876a-4edd-8ac5-52698dfd24f8\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " Jul 02 19:08:46.024270 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.024246 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-oauth-serving-cert\") pod \"691855e5-876a-4edd-8ac5-52698dfd24f8\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " Jul 02 19:08:46.024476 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.024279 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-service-ca\") pod \"691855e5-876a-4edd-8ac5-52698dfd24f8\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " Jul 02 19:08:46.024476 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.024311 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-console-config\") pod \"691855e5-876a-4edd-8ac5-52698dfd24f8\" (UID: \"691855e5-876a-4edd-8ac5-52698dfd24f8\") " Jul 02 19:08:46.024802 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.024767 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-service-ca" (OuterVolumeSpecName: "service-ca") pod "691855e5-876a-4edd-8ac5-52698dfd24f8" (UID: "691855e5-876a-4edd-8ac5-52698dfd24f8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:08:46.024905 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.024812 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "691855e5-876a-4edd-8ac5-52698dfd24f8" (UID: "691855e5-876a-4edd-8ac5-52698dfd24f8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:08:46.024905 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.024845 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "691855e5-876a-4edd-8ac5-52698dfd24f8" (UID: "691855e5-876a-4edd-8ac5-52698dfd24f8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:08:46.024905 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.024877 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-console-config" (OuterVolumeSpecName: "console-config") pod "691855e5-876a-4edd-8ac5-52698dfd24f8" (UID: "691855e5-876a-4edd-8ac5-52698dfd24f8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:08:46.026262 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.026242 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/691855e5-876a-4edd-8ac5-52698dfd24f8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "691855e5-876a-4edd-8ac5-52698dfd24f8" (UID: "691855e5-876a-4edd-8ac5-52698dfd24f8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:08:46.026761 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.026727 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/691855e5-876a-4edd-8ac5-52698dfd24f8-kube-api-access-pz8s5" (OuterVolumeSpecName: "kube-api-access-pz8s5") pod "691855e5-876a-4edd-8ac5-52698dfd24f8" (UID: "691855e5-876a-4edd-8ac5-52698dfd24f8"). InnerVolumeSpecName "kube-api-access-pz8s5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:08:46.026855 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.026763 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/691855e5-876a-4edd-8ac5-52698dfd24f8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "691855e5-876a-4edd-8ac5-52698dfd24f8" (UID: "691855e5-876a-4edd-8ac5-52698dfd24f8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:08:46.125381 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.125331 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-oauth-serving-cert\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:08:46.125381 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.125374 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-service-ca\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:08:46.125381 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.125385 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-console-config\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:08:46.125381 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.125393 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/691855e5-876a-4edd-8ac5-52698dfd24f8-console-serving-cert\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:08:46.125381 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.125402 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/691855e5-876a-4edd-8ac5-52698dfd24f8-console-oauth-config\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:08:46.125704 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.125411 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pz8s5\" (UniqueName: \"kubernetes.io/projected/691855e5-876a-4edd-8ac5-52698dfd24f8-kube-api-access-pz8s5\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:08:46.125704 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.125419 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/691855e5-876a-4edd-8ac5-52698dfd24f8-trusted-ca-bundle\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:08:46.703394 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.703363 2566 generic.go:356] "Generic (PLEG): container finished" podID="691855e5-876a-4edd-8ac5-52698dfd24f8" containerID="3dc7f05da1b93cc02b9e95bda687a582c76fa4a4a8e9d6b85b8cbde0e2c16f69" exitCode=0 Jul 02 19:08:46.703839 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.703406 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c78c9564-lzlgp" Jul 02 19:08:46.703839 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.703436 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c78c9564-lzlgp" event={"ID":"691855e5-876a-4edd-8ac5-52698dfd24f8","Type":"ContainerDied","Data":"3dc7f05da1b93cc02b9e95bda687a582c76fa4a4a8e9d6b85b8cbde0e2c16f69"} Jul 02 19:08:46.703839 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.703475 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c78c9564-lzlgp" event={"ID":"691855e5-876a-4edd-8ac5-52698dfd24f8","Type":"ContainerDied","Data":"e32b0345c163c74237cdd1ffc48b07272c755aa8b24499e09bf1d8e88485d701"} Jul 02 19:08:46.703839 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.703494 2566 scope.go:117] "RemoveContainer" containerID="3dc7f05da1b93cc02b9e95bda687a582c76fa4a4a8e9d6b85b8cbde0e2c16f69" Jul 02 19:08:46.711876 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.711861 2566 scope.go:117] "RemoveContainer" containerID="3dc7f05da1b93cc02b9e95bda687a582c76fa4a4a8e9d6b85b8cbde0e2c16f69" Jul 02 19:08:46.712130 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:08:46.712112 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dc7f05da1b93cc02b9e95bda687a582c76fa4a4a8e9d6b85b8cbde0e2c16f69\": container with ID starting with 3dc7f05da1b93cc02b9e95bda687a582c76fa4a4a8e9d6b85b8cbde0e2c16f69 not found: ID does not exist" containerID="3dc7f05da1b93cc02b9e95bda687a582c76fa4a4a8e9d6b85b8cbde0e2c16f69" Jul 02 19:08:46.712174 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.712138 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dc7f05da1b93cc02b9e95bda687a582c76fa4a4a8e9d6b85b8cbde0e2c16f69"} err="failed to get container status \"3dc7f05da1b93cc02b9e95bda687a582c76fa4a4a8e9d6b85b8cbde0e2c16f69\": rpc error: code = NotFound desc = could not find container \"3dc7f05da1b93cc02b9e95bda687a582c76fa4a4a8e9d6b85b8cbde0e2c16f69\": container with ID starting with 3dc7f05da1b93cc02b9e95bda687a582c76fa4a4a8e9d6b85b8cbde0e2c16f69 not found: ID does not exist" Jul 02 19:08:46.728251 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.728221 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86c78c9564-lzlgp"] Jul 02 19:08:46.741054 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.741026 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86c78c9564-lzlgp"] Jul 02 19:08:46.842841 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:46.842811 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="691855e5-876a-4edd-8ac5-52698dfd24f8" path="/var/lib/kubelet/pods/691855e5-876a-4edd-8ac5-52698dfd24f8/volumes" Jul 02 19:08:55.356243 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:55.356205 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:08:55.409831 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:08:55.409802 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:09:05.143545 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:05.143506 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:09:05.244910 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:05.244878 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:09:15.095080 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:15.095048 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:09:15.145802 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:15.145755 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:09:15.618455 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:15.618420 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:09:16.147119 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.147079 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c57cb8c5-hkmfc"] Jul 02 19:09:16.147556 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.147375 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="691855e5-876a-4edd-8ac5-52698dfd24f8" containerName="console" Jul 02 19:09:16.147556 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.147390 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="691855e5-876a-4edd-8ac5-52698dfd24f8" containerName="console" Jul 02 19:09:16.147556 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.147480 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="691855e5-876a-4edd-8ac5-52698dfd24f8" containerName="console" Jul 02 19:09:16.149208 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.149187 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.164120 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.164090 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c57cb8c5-hkmfc"] Jul 02 19:09:16.201734 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.201703 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:09:16.257645 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.257610 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-console-serving-cert\") pod \"console-c57cb8c5-hkmfc\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.257852 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.257654 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfz9r\" (UniqueName: \"kubernetes.io/projected/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-kube-api-access-vfz9r\") pod \"console-c57cb8c5-hkmfc\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.257852 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.257710 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-console-oauth-config\") pod \"console-c57cb8c5-hkmfc\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.257852 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.257766 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-service-ca\") pod \"console-c57cb8c5-hkmfc\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.257852 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.257804 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-trusted-ca-bundle\") pod \"console-c57cb8c5-hkmfc\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.258034 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.257862 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-console-config\") pod \"console-c57cb8c5-hkmfc\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.258034 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.257889 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-oauth-serving-cert\") pod \"console-c57cb8c5-hkmfc\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.258406 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.258381 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:09:16.341843 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.341800 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:09:16.358973 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.358942 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-oauth-serving-cert\") pod \"console-c57cb8c5-hkmfc\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.359097 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.358987 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-console-serving-cert\") pod \"console-c57cb8c5-hkmfc\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.359097 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.359024 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfz9r\" (UniqueName: \"kubernetes.io/projected/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-kube-api-access-vfz9r\") pod \"console-c57cb8c5-hkmfc\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.359097 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.359063 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-console-oauth-config\") pod \"console-c57cb8c5-hkmfc\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.359221 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.359097 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-service-ca\") pod \"console-c57cb8c5-hkmfc\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.359221 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.359141 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-trusted-ca-bundle\") pod \"console-c57cb8c5-hkmfc\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.359221 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.359176 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-console-config\") pod \"console-c57cb8c5-hkmfc\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.359876 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.359849 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-oauth-serving-cert\") pod \"console-c57cb8c5-hkmfc\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.360016 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.359850 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-console-config\") pod \"console-c57cb8c5-hkmfc\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.360016 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.359932 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-service-ca\") pod \"console-c57cb8c5-hkmfc\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.360130 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.360024 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-trusted-ca-bundle\") pod \"console-c57cb8c5-hkmfc\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.361534 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.361508 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-console-serving-cert\") pod \"console-c57cb8c5-hkmfc\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.361661 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.361642 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-console-oauth-config\") pod \"console-c57cb8c5-hkmfc\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.366439 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.366416 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfz9r\" (UniqueName: \"kubernetes.io/projected/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-kube-api-access-vfz9r\") pod \"console-c57cb8c5-hkmfc\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.458139 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.458048 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:16.586224 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.586194 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c57cb8c5-hkmfc"] Jul 02 19:09:16.588572 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:09:16.588542 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb7a8d73_8799_4204_bb9d_f9d730abf9a9.slice/crio-7bc46b58702338452ea62c24cd6691e1a707b5497dfe5dca2f7fff389cd923b9 WatchSource:0}: Error finding container 7bc46b58702338452ea62c24cd6691e1a707b5497dfe5dca2f7fff389cd923b9: Status 404 returned error can't find the container with id 7bc46b58702338452ea62c24cd6691e1a707b5497dfe5dca2f7fff389cd923b9 Jul 02 19:09:16.812971 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.812935 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c57cb8c5-hkmfc" event={"ID":"eb7a8d73-8799-4204-bb9d-f9d730abf9a9","Type":"ContainerStarted","Data":"dddc262194707b90a868db82f3e7c781c485f3d61ecfb38757dce09aa12aa31e"} Jul 02 19:09:16.813155 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.812990 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c57cb8c5-hkmfc" event={"ID":"eb7a8d73-8799-4204-bb9d-f9d730abf9a9","Type":"ContainerStarted","Data":"7bc46b58702338452ea62c24cd6691e1a707b5497dfe5dca2f7fff389cd923b9"} Jul 02 19:09:16.832017 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.831952 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c57cb8c5-hkmfc" podStartSLOduration=0.831932282 podStartE2EDuration="831.932282ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:09:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:09:16.829824989 +0000 UTC m=+236.756833701" watchObservedRunningTime="2026-07-02 19:09:16.831932282 +0000 UTC m=+236.758940996" Jul 02 19:09:16.948401 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:16.948364 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:09:17.001797 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:17.001764 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:09:26.458739 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:26.458702 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:26.459173 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:26.458763 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:26.463527 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:26.463502 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:26.852968 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:26.852940 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:09:26.899141 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:26.899106 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b64fc4db7-kvx8b"] Jul 02 19:09:27.016307 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:27.016275 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:09:27.117445 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:27.117364 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:09:27.217865 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:27.217827 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:09:40.043252 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:40.043220 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:09:44.414695 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:44.414577 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:09:44.517546 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:44.517517 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:09:51.926070 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:51.926026 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6b64fc4db7-kvx8b" podUID="2ceec284-1c1e-4882-879d-ca920b9f23f9" containerName="console" containerID="cri-o://1521acab9bd8bfc1123d9e25c196d67094f9441cf1fdb5b08e621124aa003617" gracePeriod=15 Jul 02 19:09:52.161200 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.161175 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:09:52.270704 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.270657 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-oauth-serving-cert\") pod \"2ceec284-1c1e-4882-879d-ca920b9f23f9\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " Jul 02 19:09:52.270869 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.270714 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-trusted-ca-bundle\") pod \"2ceec284-1c1e-4882-879d-ca920b9f23f9\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " Jul 02 19:09:52.270869 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.270744 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-service-ca\") pod \"2ceec284-1c1e-4882-879d-ca920b9f23f9\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " Jul 02 19:09:52.270869 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.270785 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ceec284-1c1e-4882-879d-ca920b9f23f9-console-oauth-config\") pod \"2ceec284-1c1e-4882-879d-ca920b9f23f9\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " Jul 02 19:09:52.270869 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.270818 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-console-config\") pod \"2ceec284-1c1e-4882-879d-ca920b9f23f9\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " Jul 02 19:09:52.270869 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.270847 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ceec284-1c1e-4882-879d-ca920b9f23f9-console-serving-cert\") pod \"2ceec284-1c1e-4882-879d-ca920b9f23f9\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " Jul 02 19:09:52.271124 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.270896 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdj6q\" (UniqueName: \"kubernetes.io/projected/2ceec284-1c1e-4882-879d-ca920b9f23f9-kube-api-access-mdj6q\") pod \"2ceec284-1c1e-4882-879d-ca920b9f23f9\" (UID: \"2ceec284-1c1e-4882-879d-ca920b9f23f9\") " Jul 02 19:09:52.271124 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.271018 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2ceec284-1c1e-4882-879d-ca920b9f23f9" (UID: "2ceec284-1c1e-4882-879d-ca920b9f23f9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:09:52.271286 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.271145 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-oauth-serving-cert\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:09:52.271286 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.271186 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-service-ca" (OuterVolumeSpecName: "service-ca") pod "2ceec284-1c1e-4882-879d-ca920b9f23f9" (UID: "2ceec284-1c1e-4882-879d-ca920b9f23f9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:09:52.271286 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.271214 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2ceec284-1c1e-4882-879d-ca920b9f23f9" (UID: "2ceec284-1c1e-4882-879d-ca920b9f23f9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:09:52.271542 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.271512 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-console-config" (OuterVolumeSpecName: "console-config") pod "2ceec284-1c1e-4882-879d-ca920b9f23f9" (UID: "2ceec284-1c1e-4882-879d-ca920b9f23f9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:09:52.272979 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.272953 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ceec284-1c1e-4882-879d-ca920b9f23f9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2ceec284-1c1e-4882-879d-ca920b9f23f9" (UID: "2ceec284-1c1e-4882-879d-ca920b9f23f9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:09:52.273079 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.272996 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ceec284-1c1e-4882-879d-ca920b9f23f9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2ceec284-1c1e-4882-879d-ca920b9f23f9" (UID: "2ceec284-1c1e-4882-879d-ca920b9f23f9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:09:52.273079 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.273008 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ceec284-1c1e-4882-879d-ca920b9f23f9-kube-api-access-mdj6q" (OuterVolumeSpecName: "kube-api-access-mdj6q") pod "2ceec284-1c1e-4882-879d-ca920b9f23f9" (UID: "2ceec284-1c1e-4882-879d-ca920b9f23f9"). InnerVolumeSpecName "kube-api-access-mdj6q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:09:52.371976 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.371944 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-trusted-ca-bundle\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:09:52.371976 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.371973 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-service-ca\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:09:52.371976 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.371982 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ceec284-1c1e-4882-879d-ca920b9f23f9-console-oauth-config\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:09:52.372196 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.371991 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ceec284-1c1e-4882-879d-ca920b9f23f9-console-config\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:09:52.372196 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.372000 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ceec284-1c1e-4882-879d-ca920b9f23f9-console-serving-cert\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:09:52.372196 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.372008 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdj6q\" (UniqueName: \"kubernetes.io/projected/2ceec284-1c1e-4882-879d-ca920b9f23f9-kube-api-access-mdj6q\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:09:52.941193 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.941159 2566 generic.go:356] "Generic (PLEG): container finished" podID="2ceec284-1c1e-4882-879d-ca920b9f23f9" containerID="1521acab9bd8bfc1123d9e25c196d67094f9441cf1fdb5b08e621124aa003617" exitCode=0 Jul 02 19:09:52.941584 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.941209 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b64fc4db7-kvx8b" Jul 02 19:09:52.941584 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.941249 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b64fc4db7-kvx8b" event={"ID":"2ceec284-1c1e-4882-879d-ca920b9f23f9","Type":"ContainerDied","Data":"1521acab9bd8bfc1123d9e25c196d67094f9441cf1fdb5b08e621124aa003617"} Jul 02 19:09:52.941584 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.941300 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b64fc4db7-kvx8b" event={"ID":"2ceec284-1c1e-4882-879d-ca920b9f23f9","Type":"ContainerDied","Data":"30099da7465965235e9d018c0a2deabfe016207e9570c5a32ed12023407bebdb"} Jul 02 19:09:52.941584 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.941323 2566 scope.go:117] "RemoveContainer" containerID="1521acab9bd8bfc1123d9e25c196d67094f9441cf1fdb5b08e621124aa003617" Jul 02 19:09:52.949159 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.949142 2566 scope.go:117] "RemoveContainer" containerID="1521acab9bd8bfc1123d9e25c196d67094f9441cf1fdb5b08e621124aa003617" Jul 02 19:09:52.949431 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:09:52.949414 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1521acab9bd8bfc1123d9e25c196d67094f9441cf1fdb5b08e621124aa003617\": container with ID starting with 1521acab9bd8bfc1123d9e25c196d67094f9441cf1fdb5b08e621124aa003617 not found: ID does not exist" containerID="1521acab9bd8bfc1123d9e25c196d67094f9441cf1fdb5b08e621124aa003617" Jul 02 19:09:52.949477 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.949440 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1521acab9bd8bfc1123d9e25c196d67094f9441cf1fdb5b08e621124aa003617"} err="failed to get container status \"1521acab9bd8bfc1123d9e25c196d67094f9441cf1fdb5b08e621124aa003617\": rpc error: code = NotFound desc = could not find container \"1521acab9bd8bfc1123d9e25c196d67094f9441cf1fdb5b08e621124aa003617\": container with ID starting with 1521acab9bd8bfc1123d9e25c196d67094f9441cf1fdb5b08e621124aa003617 not found: ID does not exist" Jul 02 19:09:52.956896 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.956873 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b64fc4db7-kvx8b"] Jul 02 19:09:52.960708 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:52.960667 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b64fc4db7-kvx8b"] Jul 02 19:09:54.842743 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:09:54.842713 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ceec284-1c1e-4882-879d-ca920b9f23f9" path="/var/lib/kubelet/pods/2ceec284-1c1e-4882-879d-ca920b9f23f9/volumes" Jul 02 19:10:20.059033 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:20.058990 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:10:20.691136 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:20.691109 2566 kubelet.go:1648] "Image garbage collection succeeded" Jul 02 19:10:20.709884 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:20.709857 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz"] Jul 02 19:10:20.710206 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:20.710190 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ceec284-1c1e-4882-879d-ca920b9f23f9" containerName="console" Jul 02 19:10:20.710286 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:20.710209 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ceec284-1c1e-4882-879d-ca920b9f23f9" containerName="console" Jul 02 19:10:20.710286 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:20.710271 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ceec284-1c1e-4882-879d-ca920b9f23f9" containerName="console" Jul 02 19:10:20.712897 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:20.712876 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz" Jul 02 19:10:20.715313 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:20.715292 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 19:10:20.715413 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:20.715323 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 19:10:20.716197 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:20.716183 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tnws7\"" Jul 02 19:10:20.723368 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:20.723348 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz"] Jul 02 19:10:20.787734 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:20.787701 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rzf5\" (UniqueName: \"kubernetes.io/projected/7629eb6c-abfb-47d0-94c5-06c96398e070-kube-api-access-8rzf5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz\" (UID: \"7629eb6c-abfb-47d0-94c5-06c96398e070\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz" Jul 02 19:10:20.787908 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:20.787761 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7629eb6c-abfb-47d0-94c5-06c96398e070-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz\" (UID: \"7629eb6c-abfb-47d0-94c5-06c96398e070\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz" Jul 02 19:10:20.787908 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:20.787838 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7629eb6c-abfb-47d0-94c5-06c96398e070-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz\" (UID: \"7629eb6c-abfb-47d0-94c5-06c96398e070\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz" Jul 02 19:10:20.888461 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:20.888274 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7629eb6c-abfb-47d0-94c5-06c96398e070-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz\" (UID: \"7629eb6c-abfb-47d0-94c5-06c96398e070\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz" Jul 02 19:10:20.888461 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:20.888341 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rzf5\" (UniqueName: \"kubernetes.io/projected/7629eb6c-abfb-47d0-94c5-06c96398e070-kube-api-access-8rzf5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz\" (UID: \"7629eb6c-abfb-47d0-94c5-06c96398e070\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz" Jul 02 19:10:20.888461 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:20.888376 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7629eb6c-abfb-47d0-94c5-06c96398e070-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz\" (UID: \"7629eb6c-abfb-47d0-94c5-06c96398e070\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz" Jul 02 19:10:20.893060 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:20.888766 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7629eb6c-abfb-47d0-94c5-06c96398e070-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz\" (UID: \"7629eb6c-abfb-47d0-94c5-06c96398e070\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz" Jul 02 19:10:20.893060 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:20.888777 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7629eb6c-abfb-47d0-94c5-06c96398e070-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz\" (UID: \"7629eb6c-abfb-47d0-94c5-06c96398e070\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz" Jul 02 19:10:20.897293 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:20.897274 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rzf5\" (UniqueName: \"kubernetes.io/projected/7629eb6c-abfb-47d0-94c5-06c96398e070-kube-api-access-8rzf5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz\" (UID: \"7629eb6c-abfb-47d0-94c5-06c96398e070\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz" Jul 02 19:10:21.022120 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:21.022090 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz" Jul 02 19:10:21.148757 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:21.148727 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz"] Jul 02 19:10:21.152013 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:10:21.151985 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7629eb6c_abfb_47d0_94c5_06c96398e070.slice/crio-3a5f86ca446f2078742c3092548d59b0be2e17dae0487f9a7c8c09431cae331c WatchSource:0}: Error finding container 3a5f86ca446f2078742c3092548d59b0be2e17dae0487f9a7c8c09431cae331c: Status 404 returned error can't find the container with id 3a5f86ca446f2078742c3092548d59b0be2e17dae0487f9a7c8c09431cae331c Jul 02 19:10:21.476807 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:21.476780 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:10:22.046184 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:22.046146 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz" event={"ID":"7629eb6c-abfb-47d0-94c5-06c96398e070","Type":"ContainerStarted","Data":"3a5f86ca446f2078742c3092548d59b0be2e17dae0487f9a7c8c09431cae331c"} Jul 02 19:10:29.074377 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:29.074342 2566 generic.go:356] "Generic (PLEG): container finished" podID="7629eb6c-abfb-47d0-94c5-06c96398e070" containerID="7c2840d36197fb7c7233f1250fd93430f1a64dc0d7356ced28d2464fb020dedd" exitCode=0 Jul 02 19:10:29.074804 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:29.074386 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz" event={"ID":"7629eb6c-abfb-47d0-94c5-06c96398e070","Type":"ContainerDied","Data":"7c2840d36197fb7c7233f1250fd93430f1a64dc0d7356ced28d2464fb020dedd"} Jul 02 19:10:29.075377 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:29.075360 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 19:10:33.093776 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:33.093740 2566 generic.go:356] "Generic (PLEG): container finished" podID="7629eb6c-abfb-47d0-94c5-06c96398e070" containerID="cbceb4955c1752bbfb04541b72ffbe7e96fed7e3223a6637c2e55059834df13e" exitCode=0 Jul 02 19:10:33.094163 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:33.093824 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz" event={"ID":"7629eb6c-abfb-47d0-94c5-06c96398e070","Type":"ContainerDied","Data":"cbceb4955c1752bbfb04541b72ffbe7e96fed7e3223a6637c2e55059834df13e"} Jul 02 19:10:41.131523 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:41.131488 2566 generic.go:356] "Generic (PLEG): container finished" podID="7629eb6c-abfb-47d0-94c5-06c96398e070" containerID="b7d25e8b6cbfdc279964a5dc8ad108c996fc763e4261f7abce76d7ce0c3fd2fd" exitCode=0 Jul 02 19:10:41.131984 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:41.131571 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz" event={"ID":"7629eb6c-abfb-47d0-94c5-06c96398e070","Type":"ContainerDied","Data":"b7d25e8b6cbfdc279964a5dc8ad108c996fc763e4261f7abce76d7ce0c3fd2fd"} Jul 02 19:10:42.258375 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:42.258348 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz" Jul 02 19:10:42.371736 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:42.371669 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rzf5\" (UniqueName: \"kubernetes.io/projected/7629eb6c-abfb-47d0-94c5-06c96398e070-kube-api-access-8rzf5\") pod \"7629eb6c-abfb-47d0-94c5-06c96398e070\" (UID: \"7629eb6c-abfb-47d0-94c5-06c96398e070\") " Jul 02 19:10:42.371884 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:42.371748 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7629eb6c-abfb-47d0-94c5-06c96398e070-bundle\") pod \"7629eb6c-abfb-47d0-94c5-06c96398e070\" (UID: \"7629eb6c-abfb-47d0-94c5-06c96398e070\") " Jul 02 19:10:42.371884 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:42.371798 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7629eb6c-abfb-47d0-94c5-06c96398e070-util\") pod \"7629eb6c-abfb-47d0-94c5-06c96398e070\" (UID: \"7629eb6c-abfb-47d0-94c5-06c96398e070\") " Jul 02 19:10:42.372438 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:42.372411 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7629eb6c-abfb-47d0-94c5-06c96398e070-bundle" (OuterVolumeSpecName: "bundle") pod "7629eb6c-abfb-47d0-94c5-06c96398e070" (UID: "7629eb6c-abfb-47d0-94c5-06c96398e070"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:10:42.373942 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:42.373915 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7629eb6c-abfb-47d0-94c5-06c96398e070-kube-api-access-8rzf5" (OuterVolumeSpecName: "kube-api-access-8rzf5") pod "7629eb6c-abfb-47d0-94c5-06c96398e070" (UID: "7629eb6c-abfb-47d0-94c5-06c96398e070"). InnerVolumeSpecName "kube-api-access-8rzf5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:10:42.376351 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:42.376330 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7629eb6c-abfb-47d0-94c5-06c96398e070-util" (OuterVolumeSpecName: "util") pod "7629eb6c-abfb-47d0-94c5-06c96398e070" (UID: "7629eb6c-abfb-47d0-94c5-06c96398e070"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:10:42.473146 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:42.473057 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rzf5\" (UniqueName: \"kubernetes.io/projected/7629eb6c-abfb-47d0-94c5-06c96398e070-kube-api-access-8rzf5\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:10:42.473146 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:42.473088 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7629eb6c-abfb-47d0-94c5-06c96398e070-bundle\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:10:42.473146 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:42.473097 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7629eb6c-abfb-47d0-94c5-06c96398e070-util\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:10:43.140219 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:43.140134 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz" event={"ID":"7629eb6c-abfb-47d0-94c5-06c96398e070","Type":"ContainerDied","Data":"3a5f86ca446f2078742c3092548d59b0be2e17dae0487f9a7c8c09431cae331c"} Jul 02 19:10:43.140219 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:43.140182 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a5f86ca446f2078742c3092548d59b0be2e17dae0487f9a7c8c09431cae331c" Jul 02 19:10:43.140219 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:43.140147 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmjpz" Jul 02 19:10:46.751380 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:46.751352 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:10:47.572077 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:47.572027 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:10:48.732651 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:48.732619 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t2lzt"] Jul 02 19:10:48.733023 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:48.732942 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7629eb6c-abfb-47d0-94c5-06c96398e070" containerName="extract" Jul 02 19:10:48.733023 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:48.732953 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7629eb6c-abfb-47d0-94c5-06c96398e070" containerName="extract" Jul 02 19:10:48.733023 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:48.732962 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7629eb6c-abfb-47d0-94c5-06c96398e070" containerName="util" Jul 02 19:10:48.733023 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:48.732968 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7629eb6c-abfb-47d0-94c5-06c96398e070" containerName="util" Jul 02 19:10:48.733023 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:48.732973 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7629eb6c-abfb-47d0-94c5-06c96398e070" containerName="pull" Jul 02 19:10:48.733023 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:48.732979 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7629eb6c-abfb-47d0-94c5-06c96398e070" containerName="pull" Jul 02 19:10:48.733208 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:48.733030 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7629eb6c-abfb-47d0-94c5-06c96398e070" containerName="extract" Jul 02 19:10:48.740428 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:48.740405 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t2lzt" Jul 02 19:10:48.742984 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:48.742952 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 02 19:10:48.743151 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:48.743078 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 02 19:10:48.743151 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:48.743082 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-p6698\"" Jul 02 19:10:48.748448 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:48.748430 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t2lzt"] Jul 02 19:10:48.924079 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:48.924043 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e63c0387-7a82-4979-8e0b-9d441b58c2cc-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-t2lzt\" (UID: \"e63c0387-7a82-4979-8e0b-9d441b58c2cc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t2lzt" Jul 02 19:10:48.924251 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:48.924097 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2wsd\" (UniqueName: \"kubernetes.io/projected/e63c0387-7a82-4979-8e0b-9d441b58c2cc-kube-api-access-n2wsd\") pod \"cert-manager-operator-controller-manager-55c444bf86-t2lzt\" (UID: \"e63c0387-7a82-4979-8e0b-9d441b58c2cc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t2lzt" Jul 02 19:10:49.025016 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:49.024930 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2wsd\" (UniqueName: \"kubernetes.io/projected/e63c0387-7a82-4979-8e0b-9d441b58c2cc-kube-api-access-n2wsd\") pod \"cert-manager-operator-controller-manager-55c444bf86-t2lzt\" (UID: \"e63c0387-7a82-4979-8e0b-9d441b58c2cc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t2lzt" Jul 02 19:10:49.025016 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:49.024992 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e63c0387-7a82-4979-8e0b-9d441b58c2cc-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-t2lzt\" (UID: \"e63c0387-7a82-4979-8e0b-9d441b58c2cc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t2lzt" Jul 02 19:10:49.025416 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:49.025393 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e63c0387-7a82-4979-8e0b-9d441b58c2cc-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-t2lzt\" (UID: \"e63c0387-7a82-4979-8e0b-9d441b58c2cc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t2lzt" Jul 02 19:10:49.032876 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:49.032853 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2wsd\" (UniqueName: \"kubernetes.io/projected/e63c0387-7a82-4979-8e0b-9d441b58c2cc-kube-api-access-n2wsd\") pod \"cert-manager-operator-controller-manager-55c444bf86-t2lzt\" (UID: \"e63c0387-7a82-4979-8e0b-9d441b58c2cc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t2lzt" Jul 02 19:10:49.049787 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:49.049762 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t2lzt" Jul 02 19:10:49.181696 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:49.181653 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t2lzt"] Jul 02 19:10:49.184161 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:10:49.184133 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode63c0387_7a82_4979_8e0b_9d441b58c2cc.slice/crio-d7b6e97b592709d1787f33f4254b0e2c51d5b83712496d8cd16fc0ec4c468e2c WatchSource:0}: Error finding container d7b6e97b592709d1787f33f4254b0e2c51d5b83712496d8cd16fc0ec4c468e2c: Status 404 returned error can't find the container with id d7b6e97b592709d1787f33f4254b0e2c51d5b83712496d8cd16fc0ec4c468e2c Jul 02 19:10:50.166306 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:50.166264 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t2lzt" event={"ID":"e63c0387-7a82-4979-8e0b-9d441b58c2cc","Type":"ContainerStarted","Data":"d7b6e97b592709d1787f33f4254b0e2c51d5b83712496d8cd16fc0ec4c468e2c"} Jul 02 19:10:50.364214 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:50.364175 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:10:50.464604 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:50.464507 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:10:51.172166 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:51.172132 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t2lzt" event={"ID":"e63c0387-7a82-4979-8e0b-9d441b58c2cc","Type":"ContainerStarted","Data":"12fb85b724afaf23b9f25695e72348da6a195d759e052614b77d362c53fa03ad"} Jul 02 19:10:51.194919 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:51.194859 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t2lzt" podStartSLOduration=1.7288141110000002 podStartE2EDuration="3.194836754s" totalImagesPullingTime="1.466022643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:10:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:10:51.192158679 +0000 UTC m=+331.119167393" watchObservedRunningTime="2026-07-02 19:10:51.194836754 +0000 UTC m=+331.121845467" Jul 02 19:10:51.588727 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:51.588668 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:10:51.724928 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:51.724894 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:10:52.272643 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:52.272608 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:10:52.378107 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:52.378048 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:10:54.496242 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:54.496206 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:10:54.675432 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:54.675400 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:10:57.413009 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:57.412974 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:10:57.478057 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:57.478030 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:10:59.462089 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:59.462002 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-k2vln"] Jul 02 19:10:59.464375 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:59.464352 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-k2vln" Jul 02 19:10:59.468884 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:59.468855 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-h2xt4\"" Jul 02 19:10:59.469060 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:59.469039 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 02 19:10:59.469257 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:59.469240 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 02 19:10:59.482743 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:59.482717 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-k2vln"] Jul 02 19:10:59.506248 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:59.506222 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stqg6\" (UniqueName: \"kubernetes.io/projected/16cc6d3e-6b0f-4eb4-84d5-e00727a2350d-kube-api-access-stqg6\") pod \"cert-manager-webhook-6486ffd997-k2vln\" (UID: \"16cc6d3e-6b0f-4eb4-84d5-e00727a2350d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k2vln" Jul 02 19:10:59.506370 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:59.506259 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16cc6d3e-6b0f-4eb4-84d5-e00727a2350d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-k2vln\" (UID: \"16cc6d3e-6b0f-4eb4-84d5-e00727a2350d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k2vln" Jul 02 19:10:59.607282 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:59.607247 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stqg6\" (UniqueName: \"kubernetes.io/projected/16cc6d3e-6b0f-4eb4-84d5-e00727a2350d-kube-api-access-stqg6\") pod \"cert-manager-webhook-6486ffd997-k2vln\" (UID: \"16cc6d3e-6b0f-4eb4-84d5-e00727a2350d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k2vln" Jul 02 19:10:59.607282 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:59.607284 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16cc6d3e-6b0f-4eb4-84d5-e00727a2350d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-k2vln\" (UID: \"16cc6d3e-6b0f-4eb4-84d5-e00727a2350d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k2vln" Jul 02 19:10:59.618158 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:59.618124 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stqg6\" (UniqueName: \"kubernetes.io/projected/16cc6d3e-6b0f-4eb4-84d5-e00727a2350d-kube-api-access-stqg6\") pod \"cert-manager-webhook-6486ffd997-k2vln\" (UID: \"16cc6d3e-6b0f-4eb4-84d5-e00727a2350d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k2vln" Jul 02 19:10:59.619298 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:59.619280 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16cc6d3e-6b0f-4eb4-84d5-e00727a2350d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-k2vln\" (UID: \"16cc6d3e-6b0f-4eb4-84d5-e00727a2350d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k2vln" Jul 02 19:10:59.773230 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:59.773196 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-k2vln" Jul 02 19:10:59.914316 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:10:59.914289 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-k2vln"] Jul 02 19:10:59.918308 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:10:59.918285 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16cc6d3e_6b0f_4eb4_84d5_e00727a2350d.slice/crio-449ba8b3dc00f54233d9d02ca535536cda1d37a7b9d11b2b5371e84f0abf394a WatchSource:0}: Error finding container 449ba8b3dc00f54233d9d02ca535536cda1d37a7b9d11b2b5371e84f0abf394a: Status 404 returned error can't find the container with id 449ba8b3dc00f54233d9d02ca535536cda1d37a7b9d11b2b5371e84f0abf394a Jul 02 19:11:00.207798 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:00.207707 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-k2vln" event={"ID":"16cc6d3e-6b0f-4eb4-84d5-e00727a2350d","Type":"ContainerStarted","Data":"449ba8b3dc00f54233d9d02ca535536cda1d37a7b9d11b2b5371e84f0abf394a"} Jul 02 19:11:02.695261 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:02.695215 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:03.638128 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:03.638046 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:03.714836 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:03.714804 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:04.227087 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:04.227054 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-k2vln" event={"ID":"16cc6d3e-6b0f-4eb4-84d5-e00727a2350d","Type":"ContainerStarted","Data":"4ffa6e260980932069d00f50717a74a04cc3c16fb3413bea060a0e3b0cee4f10"} Jul 02 19:11:04.227296 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:04.227138 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-k2vln" Jul 02 19:11:04.244818 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:04.244775 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-k2vln" podStartSLOduration=1.792833424 podStartE2EDuration="5.244762387s" totalImagesPullingTime="3.451928963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:10:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:04.243864571 +0000 UTC m=+344.170873285" watchObservedRunningTime="2026-07-02 19:11:04.244762387 +0000 UTC m=+344.171771098" Jul 02 19:11:10.233461 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:10.233427 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-k2vln" Jul 02 19:11:20.309251 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:20.309171 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:20.412484 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:20.412450 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:22.156611 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:22.156579 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zns4z"] Jul 02 19:11:22.158955 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:22.158940 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zns4z" Jul 02 19:11:22.161257 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:22.161234 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 02 19:11:22.162294 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:22.162264 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 02 19:11:22.162294 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:22.162285 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-smkdf\"" Jul 02 19:11:22.167420 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:22.167398 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zns4z"] Jul 02 19:11:22.268537 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:22.268502 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b699ddd3-fa62-40c8-82a3-04e78d6ff522-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zns4z\" (UID: \"b699ddd3-fa62-40c8-82a3-04e78d6ff522\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zns4z" Jul 02 19:11:22.268723 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:22.268554 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlp9d\" (UniqueName: \"kubernetes.io/projected/b699ddd3-fa62-40c8-82a3-04e78d6ff522-kube-api-access-vlp9d\") pod \"openshift-lws-operator-fd8ccff4c-zns4z\" (UID: \"b699ddd3-fa62-40c8-82a3-04e78d6ff522\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zns4z" Jul 02 19:11:22.368998 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:22.368942 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b699ddd3-fa62-40c8-82a3-04e78d6ff522-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zns4z\" (UID: \"b699ddd3-fa62-40c8-82a3-04e78d6ff522\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zns4z" Jul 02 19:11:22.369231 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:22.369019 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlp9d\" (UniqueName: \"kubernetes.io/projected/b699ddd3-fa62-40c8-82a3-04e78d6ff522-kube-api-access-vlp9d\") pod \"openshift-lws-operator-fd8ccff4c-zns4z\" (UID: \"b699ddd3-fa62-40c8-82a3-04e78d6ff522\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zns4z" Jul 02 19:11:22.369351 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:22.369330 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b699ddd3-fa62-40c8-82a3-04e78d6ff522-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zns4z\" (UID: \"b699ddd3-fa62-40c8-82a3-04e78d6ff522\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zns4z" Jul 02 19:11:22.376942 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:22.376919 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlp9d\" (UniqueName: \"kubernetes.io/projected/b699ddd3-fa62-40c8-82a3-04e78d6ff522-kube-api-access-vlp9d\") pod \"openshift-lws-operator-fd8ccff4c-zns4z\" (UID: \"b699ddd3-fa62-40c8-82a3-04e78d6ff522\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zns4z" Jul 02 19:11:22.468253 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:22.468160 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zns4z" Jul 02 19:11:22.601559 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:22.601530 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zns4z"] Jul 02 19:11:22.603385 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:11:22.603354 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb699ddd3_fa62_40c8_82a3_04e78d6ff522.slice/crio-cefdab9f098348a8b99119ec9e19cb19e87734ecf2bb3353fa49e761486532b8 WatchSource:0}: Error finding container cefdab9f098348a8b99119ec9e19cb19e87734ecf2bb3353fa49e761486532b8: Status 404 returned error can't find the container with id cefdab9f098348a8b99119ec9e19cb19e87734ecf2bb3353fa49e761486532b8 Jul 02 19:11:23.303503 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:23.303466 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zns4z" event={"ID":"b699ddd3-fa62-40c8-82a3-04e78d6ff522","Type":"ContainerStarted","Data":"cefdab9f098348a8b99119ec9e19cb19e87734ecf2bb3353fa49e761486532b8"} Jul 02 19:11:25.314719 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:25.314659 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zns4z" event={"ID":"b699ddd3-fa62-40c8-82a3-04e78d6ff522","Type":"ContainerStarted","Data":"6d654e3848a378c213c131dd78cad31cae2c49685ac1eca86cd10a06e9102d8f"} Jul 02 19:11:25.332857 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:25.332803 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zns4z" podStartSLOduration=0.743890371 podStartE2EDuration="3.332786362s" totalImagesPullingTime="2.588895991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:25.331291828 +0000 UTC m=+365.258300540" watchObservedRunningTime="2026-07-02 19:11:25.332786362 +0000 UTC m=+365.259795075" Jul 02 19:11:26.032700 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:26.032644 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:26.131191 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:26.131160 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:27.940601 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:27.940568 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr"] Jul 02 19:11:27.943037 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:27.943021 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr" Jul 02 19:11:27.945353 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:27.945329 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 19:11:27.946183 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:27.946168 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 19:11:27.946297 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:27.946229 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tnws7\"" Jul 02 19:11:27.951634 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:27.951607 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr"] Jul 02 19:11:28.019933 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:28.019894 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/19bca0f3-d469-4247-9ebc-7602babf4ccb-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr\" (UID: \"19bca0f3-d469-4247-9ebc-7602babf4ccb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr" Jul 02 19:11:28.019933 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:28.019930 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmzvn\" (UniqueName: \"kubernetes.io/projected/19bca0f3-d469-4247-9ebc-7602babf4ccb-kube-api-access-dmzvn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr\" (UID: \"19bca0f3-d469-4247-9ebc-7602babf4ccb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr" Jul 02 19:11:28.020136 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:28.019979 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/19bca0f3-d469-4247-9ebc-7602babf4ccb-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr\" (UID: \"19bca0f3-d469-4247-9ebc-7602babf4ccb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr" Jul 02 19:11:28.120373 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:28.120334 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/19bca0f3-d469-4247-9ebc-7602babf4ccb-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr\" (UID: \"19bca0f3-d469-4247-9ebc-7602babf4ccb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr" Jul 02 19:11:28.120373 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:28.120373 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmzvn\" (UniqueName: \"kubernetes.io/projected/19bca0f3-d469-4247-9ebc-7602babf4ccb-kube-api-access-dmzvn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr\" (UID: \"19bca0f3-d469-4247-9ebc-7602babf4ccb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr" Jul 02 19:11:28.120595 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:28.120412 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/19bca0f3-d469-4247-9ebc-7602babf4ccb-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr\" (UID: \"19bca0f3-d469-4247-9ebc-7602babf4ccb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr" Jul 02 19:11:28.120833 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:28.120812 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/19bca0f3-d469-4247-9ebc-7602babf4ccb-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr\" (UID: \"19bca0f3-d469-4247-9ebc-7602babf4ccb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr" Jul 02 19:11:28.120891 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:28.120822 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/19bca0f3-d469-4247-9ebc-7602babf4ccb-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr\" (UID: \"19bca0f3-d469-4247-9ebc-7602babf4ccb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr" Jul 02 19:11:28.128729 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:28.128706 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmzvn\" (UniqueName: \"kubernetes.io/projected/19bca0f3-d469-4247-9ebc-7602babf4ccb-kube-api-access-dmzvn\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr\" (UID: \"19bca0f3-d469-4247-9ebc-7602babf4ccb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr" Jul 02 19:11:28.252434 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:28.252398 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr" Jul 02 19:11:28.350942 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:28.350910 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:28.382832 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:28.382802 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr"] Jul 02 19:11:28.386030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:11:28.386006 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19bca0f3_d469_4247_9ebc_7602babf4ccb.slice/crio-1d719ab448560f147b2bad50d9b98bf74313a51af6be2981cd1aa1b4b3b2d4a9 WatchSource:0}: Error finding container 1d719ab448560f147b2bad50d9b98bf74313a51af6be2981cd1aa1b4b3b2d4a9: Status 404 returned error can't find the container with id 1d719ab448560f147b2bad50d9b98bf74313a51af6be2981cd1aa1b4b3b2d4a9 Jul 02 19:11:29.333491 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:29.333456 2566 generic.go:356] "Generic (PLEG): container finished" podID="19bca0f3-d469-4247-9ebc-7602babf4ccb" containerID="010c638880214c1dc8f1a9913b15884812d4ef60f0ebbb70cf819563d1c23373" exitCode=0 Jul 02 19:11:29.334033 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:29.333544 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr" event={"ID":"19bca0f3-d469-4247-9ebc-7602babf4ccb","Type":"ContainerDied","Data":"010c638880214c1dc8f1a9913b15884812d4ef60f0ebbb70cf819563d1c23373"} Jul 02 19:11:29.334033 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:29.333597 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr" event={"ID":"19bca0f3-d469-4247-9ebc-7602babf4ccb","Type":"ContainerStarted","Data":"1d719ab448560f147b2bad50d9b98bf74313a51af6be2981cd1aa1b4b3b2d4a9"} Jul 02 19:11:30.339987 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:30.339956 2566 generic.go:356] "Generic (PLEG): container finished" podID="19bca0f3-d469-4247-9ebc-7602babf4ccb" containerID="a308632b8fd83b0e2d3b2e6b4b4c2993aa64c6d5ea4fd6475b8bf0f5c329463c" exitCode=0 Jul 02 19:11:30.340367 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:30.340003 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr" event={"ID":"19bca0f3-d469-4247-9ebc-7602babf4ccb","Type":"ContainerDied","Data":"a308632b8fd83b0e2d3b2e6b4b4c2993aa64c6d5ea4fd6475b8bf0f5c329463c"} Jul 02 19:11:31.346220 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:31.346181 2566 generic.go:356] "Generic (PLEG): container finished" podID="19bca0f3-d469-4247-9ebc-7602babf4ccb" containerID="57f421a3fc3a9e85c0726e631996d05711d70ddeb0c85d2316767bc28dec4e34" exitCode=0 Jul 02 19:11:31.346671 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:31.346267 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr" event={"ID":"19bca0f3-d469-4247-9ebc-7602babf4ccb","Type":"ContainerDied","Data":"57f421a3fc3a9e85c0726e631996d05711d70ddeb0c85d2316767bc28dec4e34"} Jul 02 19:11:32.473439 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:32.473414 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr" Jul 02 19:11:32.555362 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:32.555332 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/19bca0f3-d469-4247-9ebc-7602babf4ccb-bundle\") pod \"19bca0f3-d469-4247-9ebc-7602babf4ccb\" (UID: \"19bca0f3-d469-4247-9ebc-7602babf4ccb\") " Jul 02 19:11:32.555529 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:32.555414 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/19bca0f3-d469-4247-9ebc-7602babf4ccb-util\") pod \"19bca0f3-d469-4247-9ebc-7602babf4ccb\" (UID: \"19bca0f3-d469-4247-9ebc-7602babf4ccb\") " Jul 02 19:11:32.555529 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:32.555435 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmzvn\" (UniqueName: \"kubernetes.io/projected/19bca0f3-d469-4247-9ebc-7602babf4ccb-kube-api-access-dmzvn\") pod \"19bca0f3-d469-4247-9ebc-7602babf4ccb\" (UID: \"19bca0f3-d469-4247-9ebc-7602babf4ccb\") " Jul 02 19:11:32.556061 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:32.556032 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19bca0f3-d469-4247-9ebc-7602babf4ccb-bundle" (OuterVolumeSpecName: "bundle") pod "19bca0f3-d469-4247-9ebc-7602babf4ccb" (UID: "19bca0f3-d469-4247-9ebc-7602babf4ccb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:11:32.557540 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:32.557512 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19bca0f3-d469-4247-9ebc-7602babf4ccb-kube-api-access-dmzvn" (OuterVolumeSpecName: "kube-api-access-dmzvn") pod "19bca0f3-d469-4247-9ebc-7602babf4ccb" (UID: "19bca0f3-d469-4247-9ebc-7602babf4ccb"). InnerVolumeSpecName "kube-api-access-dmzvn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:11:32.560841 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:32.560821 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19bca0f3-d469-4247-9ebc-7602babf4ccb-util" (OuterVolumeSpecName: "util") pod "19bca0f3-d469-4247-9ebc-7602babf4ccb" (UID: "19bca0f3-d469-4247-9ebc-7602babf4ccb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:11:32.656360 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:32.656276 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/19bca0f3-d469-4247-9ebc-7602babf4ccb-util\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:11:32.656360 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:32.656310 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmzvn\" (UniqueName: \"kubernetes.io/projected/19bca0f3-d469-4247-9ebc-7602babf4ccb-kube-api-access-dmzvn\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:11:32.656360 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:32.656319 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/19bca0f3-d469-4247-9ebc-7602babf4ccb-bundle\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:11:33.356063 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:33.356035 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr" Jul 02 19:11:33.356211 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:33.356034 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6z6rr" event={"ID":"19bca0f3-d469-4247-9ebc-7602babf4ccb","Type":"ContainerDied","Data":"1d719ab448560f147b2bad50d9b98bf74313a51af6be2981cd1aa1b4b3b2d4a9"} Jul 02 19:11:33.356211 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:33.356157 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d719ab448560f147b2bad50d9b98bf74313a51af6be2981cd1aa1b4b3b2d4a9" Jul 02 19:11:34.624733 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.624660 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:34.691206 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.691172 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572"] Jul 02 19:11:34.691497 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.691484 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19bca0f3-d469-4247-9ebc-7602babf4ccb" containerName="extract" Jul 02 19:11:34.691497 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.691498 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="19bca0f3-d469-4247-9ebc-7602babf4ccb" containerName="extract" Jul 02 19:11:34.691583 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.691516 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19bca0f3-d469-4247-9ebc-7602babf4ccb" containerName="util" Jul 02 19:11:34.691583 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.691522 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="19bca0f3-d469-4247-9ebc-7602babf4ccb" containerName="util" Jul 02 19:11:34.691583 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.691534 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19bca0f3-d469-4247-9ebc-7602babf4ccb" containerName="pull" Jul 02 19:11:34.691583 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.691540 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="19bca0f3-d469-4247-9ebc-7602babf4ccb" containerName="pull" Jul 02 19:11:34.691740 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.691585 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="19bca0f3-d469-4247-9ebc-7602babf4ccb" containerName="extract" Jul 02 19:11:34.694662 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.694646 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572" Jul 02 19:11:34.700171 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.700129 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-snvkg\"" Jul 02 19:11:34.700307 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.700173 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 02 19:11:34.700307 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.700219 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 02 19:11:34.700307 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.700267 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 02 19:11:34.711418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.711389 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572"] Jul 02 19:11:34.718199 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.718177 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:34.774408 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.774380 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c997ebe2-d60c-4d2b-8e1a-e0d0908ea978-cert\") pod \"lws-controller-manager-5556b8b6b-qb572\" (UID: \"c997ebe2-d60c-4d2b-8e1a-e0d0908ea978\") " pod="openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572" Jul 02 19:11:34.774570 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.774415 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c997ebe2-d60c-4d2b-8e1a-e0d0908ea978-metrics-cert\") pod \"lws-controller-manager-5556b8b6b-qb572\" (UID: \"c997ebe2-d60c-4d2b-8e1a-e0d0908ea978\") " pod="openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572" Jul 02 19:11:34.774570 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.774444 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlb75\" (UniqueName: \"kubernetes.io/projected/c997ebe2-d60c-4d2b-8e1a-e0d0908ea978-kube-api-access-rlb75\") pod \"lws-controller-manager-5556b8b6b-qb572\" (UID: \"c997ebe2-d60c-4d2b-8e1a-e0d0908ea978\") " pod="openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572" Jul 02 19:11:34.774570 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.774487 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c997ebe2-d60c-4d2b-8e1a-e0d0908ea978-manager-config\") pod \"lws-controller-manager-5556b8b6b-qb572\" (UID: \"c997ebe2-d60c-4d2b-8e1a-e0d0908ea978\") " pod="openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572" Jul 02 19:11:34.875317 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.875233 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c997ebe2-d60c-4d2b-8e1a-e0d0908ea978-cert\") pod \"lws-controller-manager-5556b8b6b-qb572\" (UID: \"c997ebe2-d60c-4d2b-8e1a-e0d0908ea978\") " pod="openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572" Jul 02 19:11:34.875317 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.875273 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c997ebe2-d60c-4d2b-8e1a-e0d0908ea978-metrics-cert\") pod \"lws-controller-manager-5556b8b6b-qb572\" (UID: \"c997ebe2-d60c-4d2b-8e1a-e0d0908ea978\") " pod="openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572" Jul 02 19:11:34.875317 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.875307 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlb75\" (UniqueName: \"kubernetes.io/projected/c997ebe2-d60c-4d2b-8e1a-e0d0908ea978-kube-api-access-rlb75\") pod \"lws-controller-manager-5556b8b6b-qb572\" (UID: \"c997ebe2-d60c-4d2b-8e1a-e0d0908ea978\") " pod="openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572" Jul 02 19:11:34.875540 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.875334 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c997ebe2-d60c-4d2b-8e1a-e0d0908ea978-manager-config\") pod \"lws-controller-manager-5556b8b6b-qb572\" (UID: \"c997ebe2-d60c-4d2b-8e1a-e0d0908ea978\") " pod="openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572" Jul 02 19:11:34.876037 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.876018 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c997ebe2-d60c-4d2b-8e1a-e0d0908ea978-manager-config\") pod \"lws-controller-manager-5556b8b6b-qb572\" (UID: \"c997ebe2-d60c-4d2b-8e1a-e0d0908ea978\") " pod="openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572" Jul 02 19:11:34.877619 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.877594 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c997ebe2-d60c-4d2b-8e1a-e0d0908ea978-cert\") pod \"lws-controller-manager-5556b8b6b-qb572\" (UID: \"c997ebe2-d60c-4d2b-8e1a-e0d0908ea978\") " pod="openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572" Jul 02 19:11:34.877734 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.877661 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c997ebe2-d60c-4d2b-8e1a-e0d0908ea978-metrics-cert\") pod \"lws-controller-manager-5556b8b6b-qb572\" (UID: \"c997ebe2-d60c-4d2b-8e1a-e0d0908ea978\") " pod="openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572" Jul 02 19:11:34.884380 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:34.884359 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlb75\" (UniqueName: \"kubernetes.io/projected/c997ebe2-d60c-4d2b-8e1a-e0d0908ea978-kube-api-access-rlb75\") pod \"lws-controller-manager-5556b8b6b-qb572\" (UID: \"c997ebe2-d60c-4d2b-8e1a-e0d0908ea978\") " pod="openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572" Jul 02 19:11:35.003742 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:35.003706 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572" Jul 02 19:11:35.139980 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:35.139953 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572"] Jul 02 19:11:35.142348 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:11:35.142314 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc997ebe2_d60c_4d2b_8e1a_e0d0908ea978.slice/crio-12f29dd0777515f849d8dcd0d0fd39b10eb39852e96723b0dd349bc81c159ac7 WatchSource:0}: Error finding container 12f29dd0777515f849d8dcd0d0fd39b10eb39852e96723b0dd349bc81c159ac7: Status 404 returned error can't find the container with id 12f29dd0777515f849d8dcd0d0fd39b10eb39852e96723b0dd349bc81c159ac7 Jul 02 19:11:35.365548 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:35.365508 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572" event={"ID":"c997ebe2-d60c-4d2b-8e1a-e0d0908ea978","Type":"ContainerStarted","Data":"12f29dd0777515f849d8dcd0d0fd39b10eb39852e96723b0dd349bc81c159ac7"} Jul 02 19:11:39.384977 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:39.384920 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572" event={"ID":"c997ebe2-d60c-4d2b-8e1a-e0d0908ea978","Type":"ContainerStarted","Data":"ceb60d140026dfe8dff0287f8c3b354678e4bc1dc272acf0a711f6904537877d"} Jul 02 19:11:39.385374 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:39.385139 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572" Jul 02 19:11:39.404430 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:39.404368 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572" podStartSLOduration=1.924111585 podStartE2EDuration="5.40434415s" totalImagesPullingTime="3.480232565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:11:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:39.401105645 +0000 UTC m=+379.328114351" watchObservedRunningTime="2026-07-02 19:11:39.40434415 +0000 UTC m=+379.331352863" Jul 02 19:11:44.703982 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:44.703948 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:44.799045 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:44.799011 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:45.284606 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.284574 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:45.573731 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.573634 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:45.682939 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.682908 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:45.747518 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.743243 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5"] Jul 02 19:11:45.747938 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.747919 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" Jul 02 19:11:45.750971 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.750940 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 02 19:11:45.751131 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.750981 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 02 19:11:45.751131 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.750991 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 02 19:11:45.751390 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.751368 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 02 19:11:45.751487 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.751464 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-nmldj\"" Jul 02 19:11:45.763197 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.763174 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5"] Jul 02 19:11:45.869762 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.869654 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24127ebd-5316-48db-b150-026731d58a15-tmp\") pod \"opendatahub-operator-controller-manager-6cf784d988-ghsw5\" (UID: \"24127ebd-5316-48db-b150-026731d58a15\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" Jul 02 19:11:45.869762 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.869724 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/24127ebd-5316-48db-b150-026731d58a15-manifests\") pod \"opendatahub-operator-controller-manager-6cf784d988-ghsw5\" (UID: \"24127ebd-5316-48db-b150-026731d58a15\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" Jul 02 19:11:45.869762 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.869744 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgm8x\" (UniqueName: \"kubernetes.io/projected/24127ebd-5316-48db-b150-026731d58a15-kube-api-access-qgm8x\") pod \"opendatahub-operator-controller-manager-6cf784d988-ghsw5\" (UID: \"24127ebd-5316-48db-b150-026731d58a15\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" Jul 02 19:11:45.869992 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.869783 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/24127ebd-5316-48db-b150-026731d58a15-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cf784d988-ghsw5\" (UID: \"24127ebd-5316-48db-b150-026731d58a15\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" Jul 02 19:11:45.869992 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.869851 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/24127ebd-5316-48db-b150-026731d58a15-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cf784d988-ghsw5\" (UID: \"24127ebd-5316-48db-b150-026731d58a15\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" Jul 02 19:11:45.921772 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.921738 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:45.971192 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.971157 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24127ebd-5316-48db-b150-026731d58a15-tmp\") pod \"opendatahub-operator-controller-manager-6cf784d988-ghsw5\" (UID: \"24127ebd-5316-48db-b150-026731d58a15\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" Jul 02 19:11:45.971364 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.971204 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/24127ebd-5316-48db-b150-026731d58a15-manifests\") pod \"opendatahub-operator-controller-manager-6cf784d988-ghsw5\" (UID: \"24127ebd-5316-48db-b150-026731d58a15\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" Jul 02 19:11:45.971364 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.971231 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgm8x\" (UniqueName: \"kubernetes.io/projected/24127ebd-5316-48db-b150-026731d58a15-kube-api-access-qgm8x\") pod \"opendatahub-operator-controller-manager-6cf784d988-ghsw5\" (UID: \"24127ebd-5316-48db-b150-026731d58a15\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" Jul 02 19:11:45.971364 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.971293 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/24127ebd-5316-48db-b150-026731d58a15-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cf784d988-ghsw5\" (UID: \"24127ebd-5316-48db-b150-026731d58a15\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" Jul 02 19:11:45.971364 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.971337 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/24127ebd-5316-48db-b150-026731d58a15-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cf784d988-ghsw5\" (UID: \"24127ebd-5316-48db-b150-026731d58a15\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" Jul 02 19:11:45.971644 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.971623 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/24127ebd-5316-48db-b150-026731d58a15-manifests\") pod \"opendatahub-operator-controller-manager-6cf784d988-ghsw5\" (UID: \"24127ebd-5316-48db-b150-026731d58a15\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" Jul 02 19:11:45.973551 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.973499 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24127ebd-5316-48db-b150-026731d58a15-tmp\") pod \"opendatahub-operator-controller-manager-6cf784d988-ghsw5\" (UID: \"24127ebd-5316-48db-b150-026731d58a15\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" Jul 02 19:11:45.973755 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.973738 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/24127ebd-5316-48db-b150-026731d58a15-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cf784d988-ghsw5\" (UID: \"24127ebd-5316-48db-b150-026731d58a15\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" Jul 02 19:11:45.973856 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.973833 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/24127ebd-5316-48db-b150-026731d58a15-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cf784d988-ghsw5\" (UID: \"24127ebd-5316-48db-b150-026731d58a15\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" Jul 02 19:11:45.990927 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:45.990903 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgm8x\" (UniqueName: \"kubernetes.io/projected/24127ebd-5316-48db-b150-026731d58a15-kube-api-access-qgm8x\") pod \"opendatahub-operator-controller-manager-6cf784d988-ghsw5\" (UID: \"24127ebd-5316-48db-b150-026731d58a15\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" Jul 02 19:11:46.058936 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:46.058897 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" Jul 02 19:11:46.079382 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:46.079356 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:46.176519 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:46.176483 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:46.194164 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:46.194138 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5"] Jul 02 19:11:46.197030 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:11:46.197007 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24127ebd_5316_48db_b150_026731d58a15.slice/crio-d5fe0884d769fad41d425290175ae0cede4f79f61dc40216bffdf5842117f65e WatchSource:0}: Error finding container d5fe0884d769fad41d425290175ae0cede4f79f61dc40216bffdf5842117f65e: Status 404 returned error can't find the container with id d5fe0884d769fad41d425290175ae0cede4f79f61dc40216bffdf5842117f65e Jul 02 19:11:46.247029 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:46.246995 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:46.309828 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:46.309796 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:46.418709 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:46.418593 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" event={"ID":"24127ebd-5316-48db-b150-026731d58a15","Type":"ContainerStarted","Data":"d5fe0884d769fad41d425290175ae0cede4f79f61dc40216bffdf5842117f65e"} Jul 02 19:11:46.422196 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:46.422171 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:46.520050 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:46.520017 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:49.434834 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:49.434797 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" event={"ID":"24127ebd-5316-48db-b150-026731d58a15","Type":"ContainerStarted","Data":"8eceefb446876cd37891ed3459b87af5cd89338ca518a61cf8f0c11c31018ddc"} Jul 02 19:11:50.393150 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:50.393076 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5556b8b6b-qb572" Jul 02 19:11:50.440913 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:50.440881 2566 generic.go:356] "Generic (PLEG): container finished" podID="24127ebd-5316-48db-b150-026731d58a15" containerID="8eceefb446876cd37891ed3459b87af5cd89338ca518a61cf8f0c11c31018ddc" exitCode=0 Jul 02 19:11:50.441309 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:50.440966 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" event={"ID":"24127ebd-5316-48db-b150-026731d58a15","Type":"ContainerDied","Data":"8eceefb446876cd37891ed3459b87af5cd89338ca518a61cf8f0c11c31018ddc"} Jul 02 19:11:51.449440 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:51.449399 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" event={"ID":"24127ebd-5316-48db-b150-026731d58a15","Type":"ContainerStarted","Data":"f378b0763f2ff961c2557295f87ece36824e12abd36ab1764470f2b24ada6f24"} Jul 02 19:11:51.449943 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:51.449502 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" Jul 02 19:11:51.493026 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:51.492975 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" podStartSLOduration=2.334114514 podStartE2EDuration="6.492956545s" totalImagesPullingTime="3.148830793s" totalInitContainerRuntime="1.010011238s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:11:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:11:51.491946014 +0000 UTC m=+391.418954728" watchObservedRunningTime="2026-07-02 19:11:51.492956545 +0000 UTC m=+391.419965257" Jul 02 19:11:57.820649 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:57.820618 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:11:57.904639 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:11:57.904604 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:01.294982 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:01.294946 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:01.398775 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:01.398742 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:01.752777 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:01.752739 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:01.898971 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:01.898937 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:02.458767 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:02.458737 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6cf784d988-ghsw5" Jul 02 19:12:05.606943 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:05.606912 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:05.761465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:05.761433 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:06.545281 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:06.545230 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:06.780167 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:06.780130 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:06.930956 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:06.930869 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:07.065405 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:07.065373 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:07.703298 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:07.703268 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:08.399252 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:08.399223 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:09.234080 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:09.234040 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:09.411531 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:09.411494 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-c665485c8-s2bzz"] Jul 02 19:12:09.420969 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:09.420940 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-c665485c8-s2bzz" Jul 02 19:12:09.424389 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:09.424366 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 02 19:12:09.424628 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:09.424434 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 02 19:12:09.424829 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:09.424502 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 02 19:12:09.425067 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:09.425040 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-cb7hb\"" Jul 02 19:12:09.443539 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:09.443508 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 02 19:12:09.447378 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:09.447354 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-c665485c8-s2bzz"] Jul 02 19:12:09.468336 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:09.468310 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74-tmp\") pod \"kube-auth-proxy-c665485c8-s2bzz\" (UID: \"2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74\") " pod="openshift-ingress/kube-auth-proxy-c665485c8-s2bzz" Jul 02 19:12:09.468479 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:09.468343 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74-tls-certs\") pod \"kube-auth-proxy-c665485c8-s2bzz\" (UID: \"2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74\") " pod="openshift-ingress/kube-auth-proxy-c665485c8-s2bzz" Jul 02 19:12:09.468479 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:09.468375 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsdgf\" (UniqueName: \"kubernetes.io/projected/2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74-kube-api-access-tsdgf\") pod \"kube-auth-proxy-c665485c8-s2bzz\" (UID: \"2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74\") " pod="openshift-ingress/kube-auth-proxy-c665485c8-s2bzz" Jul 02 19:12:09.568922 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:09.568837 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsdgf\" (UniqueName: \"kubernetes.io/projected/2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74-kube-api-access-tsdgf\") pod \"kube-auth-proxy-c665485c8-s2bzz\" (UID: \"2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74\") " pod="openshift-ingress/kube-auth-proxy-c665485c8-s2bzz" Jul 02 19:12:09.569097 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:09.568928 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74-tmp\") pod \"kube-auth-proxy-c665485c8-s2bzz\" (UID: \"2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74\") " pod="openshift-ingress/kube-auth-proxy-c665485c8-s2bzz" Jul 02 19:12:09.569097 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:09.568951 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74-tls-certs\") pod \"kube-auth-proxy-c665485c8-s2bzz\" (UID: \"2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74\") " pod="openshift-ingress/kube-auth-proxy-c665485c8-s2bzz" Jul 02 19:12:09.569097 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:12:09.569038 2566 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 02 19:12:09.569266 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:12:09.569113 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74-tls-certs podName:2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74 nodeName:}" failed. No retries permitted until 2026-07-02 19:12:10.06909485 +0000 UTC m=+409.996103544 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74-tls-certs") pod "kube-auth-proxy-c665485c8-s2bzz" (UID: "2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74") : secret "kube-auth-proxy-tls" not found Jul 02 19:12:09.571196 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:09.571174 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74-tmp\") pod \"kube-auth-proxy-c665485c8-s2bzz\" (UID: \"2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74\") " pod="openshift-ingress/kube-auth-proxy-c665485c8-s2bzz" Jul 02 19:12:09.586127 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:09.586095 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsdgf\" (UniqueName: \"kubernetes.io/projected/2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74-kube-api-access-tsdgf\") pod \"kube-auth-proxy-c665485c8-s2bzz\" (UID: \"2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74\") " pod="openshift-ingress/kube-auth-proxy-c665485c8-s2bzz" Jul 02 19:12:09.664705 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:09.664647 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:10.072383 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:10.072340 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74-tls-certs\") pod \"kube-auth-proxy-c665485c8-s2bzz\" (UID: \"2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74\") " pod="openshift-ingress/kube-auth-proxy-c665485c8-s2bzz" Jul 02 19:12:10.074931 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:10.074907 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74-tls-certs\") pod \"kube-auth-proxy-c665485c8-s2bzz\" (UID: \"2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74\") " pod="openshift-ingress/kube-auth-proxy-c665485c8-s2bzz" Jul 02 19:12:10.332268 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:10.332176 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-c665485c8-s2bzz" Jul 02 19:12:10.488981 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:10.488801 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-c665485c8-s2bzz"] Jul 02 19:12:10.491588 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:12:10.491559 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d64ee9e_1fd2_4c68_aa84_d23d8afc4b74.slice/crio-9c0abc2bfcb21f10ab9b84e456e99c7eb88415da55dfb931efae1124286adbe2 WatchSource:0}: Error finding container 9c0abc2bfcb21f10ab9b84e456e99c7eb88415da55dfb931efae1124286adbe2: Status 404 returned error can't find the container with id 9c0abc2bfcb21f10ab9b84e456e99c7eb88415da55dfb931efae1124286adbe2 Jul 02 19:12:10.540872 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:10.540836 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-c665485c8-s2bzz" event={"ID":"2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74","Type":"ContainerStarted","Data":"9c0abc2bfcb21f10ab9b84e456e99c7eb88415da55dfb931efae1124286adbe2"} Jul 02 19:12:10.993937 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:10.993902 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:11.077895 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:11.077840 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:12.475044 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:12.475004 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:13.557708 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:13.557593 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-c665485c8-s2bzz" event={"ID":"2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74","Type":"ContainerStarted","Data":"db10c803c4f7b52dfb6fb128fc2264fb1d77c73def2f736b404f8ea9f7dce206"} Jul 02 19:12:13.585406 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:13.585344 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-c665485c8-s2bzz" podStartSLOduration=1.816208812 podStartE2EDuration="4.585322952s" totalImagesPullingTime="2.76911414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:12:13.582339048 +0000 UTC m=+413.509347759" watchObservedRunningTime="2026-07-02 19:12:13.585322952 +0000 UTC m=+413.512331664" Jul 02 19:12:15.217983 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:15.217946 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:15.323269 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:15.323234 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:16.403661 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:16.403615 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:16.672291 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:16.672199 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:16.956447 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:16.956360 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:17.574401 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:17.574359 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:17.753143 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:17.753097 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:17.856202 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:17.856104 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:18.523930 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:18.523892 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:18.926259 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:18.926178 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:19.038412 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:19.038377 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:19.211126 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:19.211039 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:20.882649 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:20.882614 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:37.823645 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:37.823609 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:37.884996 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:37.884966 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:49.060275 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:49.060190 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:12:49.116471 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:12:49.116435 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:07.013427 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:07.013386 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:07.124265 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:07.124228 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:07.225208 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:07.225172 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:07.925509 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:07.925473 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:08.120879 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:08.120823 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:18.481515 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.481473 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5854c674c6-czlnf"] Jul 02 19:13:18.484857 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.484833 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.497947 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.497923 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5854c674c6-czlnf"] Jul 02 19:13:18.543689 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.543642 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a59376e-4d55-42d0-8c31-0bf26d144a9f-service-ca\") pod \"console-5854c674c6-czlnf\" (UID: \"6a59376e-4d55-42d0-8c31-0bf26d144a9f\") " pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.543866 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.543705 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a59376e-4d55-42d0-8c31-0bf26d144a9f-console-oauth-config\") pod \"console-5854c674c6-czlnf\" (UID: \"6a59376e-4d55-42d0-8c31-0bf26d144a9f\") " pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.543866 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.543762 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a59376e-4d55-42d0-8c31-0bf26d144a9f-console-config\") pod \"console-5854c674c6-czlnf\" (UID: \"6a59376e-4d55-42d0-8c31-0bf26d144a9f\") " pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.543866 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.543782 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a59376e-4d55-42d0-8c31-0bf26d144a9f-trusted-ca-bundle\") pod \"console-5854c674c6-czlnf\" (UID: \"6a59376e-4d55-42d0-8c31-0bf26d144a9f\") " pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.543866 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.543804 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bjs8\" (UniqueName: \"kubernetes.io/projected/6a59376e-4d55-42d0-8c31-0bf26d144a9f-kube-api-access-8bjs8\") pod \"console-5854c674c6-czlnf\" (UID: \"6a59376e-4d55-42d0-8c31-0bf26d144a9f\") " pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.543866 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.543860 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a59376e-4d55-42d0-8c31-0bf26d144a9f-oauth-serving-cert\") pod \"console-5854c674c6-czlnf\" (UID: \"6a59376e-4d55-42d0-8c31-0bf26d144a9f\") " pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.544085 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.543885 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a59376e-4d55-42d0-8c31-0bf26d144a9f-console-serving-cert\") pod \"console-5854c674c6-czlnf\" (UID: \"6a59376e-4d55-42d0-8c31-0bf26d144a9f\") " pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.544541 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.544516 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:18.644584 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.644544 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a59376e-4d55-42d0-8c31-0bf26d144a9f-oauth-serving-cert\") pod \"console-5854c674c6-czlnf\" (UID: \"6a59376e-4d55-42d0-8c31-0bf26d144a9f\") " pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.644789 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.644593 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a59376e-4d55-42d0-8c31-0bf26d144a9f-console-serving-cert\") pod \"console-5854c674c6-czlnf\" (UID: \"6a59376e-4d55-42d0-8c31-0bf26d144a9f\") " pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.644789 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.644643 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a59376e-4d55-42d0-8c31-0bf26d144a9f-service-ca\") pod \"console-5854c674c6-czlnf\" (UID: \"6a59376e-4d55-42d0-8c31-0bf26d144a9f\") " pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.644789 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.644665 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a59376e-4d55-42d0-8c31-0bf26d144a9f-console-oauth-config\") pod \"console-5854c674c6-czlnf\" (UID: \"6a59376e-4d55-42d0-8c31-0bf26d144a9f\") " pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.644789 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.644746 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a59376e-4d55-42d0-8c31-0bf26d144a9f-console-config\") pod \"console-5854c674c6-czlnf\" (UID: \"6a59376e-4d55-42d0-8c31-0bf26d144a9f\") " pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.645010 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.644794 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a59376e-4d55-42d0-8c31-0bf26d144a9f-trusted-ca-bundle\") pod \"console-5854c674c6-czlnf\" (UID: \"6a59376e-4d55-42d0-8c31-0bf26d144a9f\") " pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.645010 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.644819 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bjs8\" (UniqueName: \"kubernetes.io/projected/6a59376e-4d55-42d0-8c31-0bf26d144a9f-kube-api-access-8bjs8\") pod \"console-5854c674c6-czlnf\" (UID: \"6a59376e-4d55-42d0-8c31-0bf26d144a9f\") " pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.645528 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.645492 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a59376e-4d55-42d0-8c31-0bf26d144a9f-oauth-serving-cert\") pod \"console-5854c674c6-czlnf\" (UID: \"6a59376e-4d55-42d0-8c31-0bf26d144a9f\") " pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.645742 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.645707 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a59376e-4d55-42d0-8c31-0bf26d144a9f-trusted-ca-bundle\") pod \"console-5854c674c6-czlnf\" (UID: \"6a59376e-4d55-42d0-8c31-0bf26d144a9f\") " pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.645854 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.645829 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a59376e-4d55-42d0-8c31-0bf26d144a9f-console-config\") pod \"console-5854c674c6-czlnf\" (UID: \"6a59376e-4d55-42d0-8c31-0bf26d144a9f\") " pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.645854 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.645829 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:18.645975 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.645883 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a59376e-4d55-42d0-8c31-0bf26d144a9f-service-ca\") pod \"console-5854c674c6-czlnf\" (UID: \"6a59376e-4d55-42d0-8c31-0bf26d144a9f\") " pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.647333 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.647306 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a59376e-4d55-42d0-8c31-0bf26d144a9f-console-oauth-config\") pod \"console-5854c674c6-czlnf\" (UID: \"6a59376e-4d55-42d0-8c31-0bf26d144a9f\") " pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.647430 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.647408 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a59376e-4d55-42d0-8c31-0bf26d144a9f-console-serving-cert\") pod \"console-5854c674c6-czlnf\" (UID: \"6a59376e-4d55-42d0-8c31-0bf26d144a9f\") " pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.652598 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.652581 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bjs8\" (UniqueName: \"kubernetes.io/projected/6a59376e-4d55-42d0-8c31-0bf26d144a9f-kube-api-access-8bjs8\") pod \"console-5854c674c6-czlnf\" (UID: \"6a59376e-4d55-42d0-8c31-0bf26d144a9f\") " pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.746435 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.746347 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:18.794829 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.794795 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:18.846587 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.846561 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:18.926995 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:18.926958 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5854c674c6-czlnf"] Jul 02 19:13:18.930020 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:13:18.929993 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a59376e_4d55_42d0_8c31_0bf26d144a9f.slice/crio-7936e03a5c8f40df8c9e774f73c69384cd5b4760deddc9530fa06d558bd7db02 WatchSource:0}: Error finding container 7936e03a5c8f40df8c9e774f73c69384cd5b4760deddc9530fa06d558bd7db02: Status 404 returned error can't find the container with id 7936e03a5c8f40df8c9e774f73c69384cd5b4760deddc9530fa06d558bd7db02 Jul 02 19:13:19.289917 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:19.289883 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:19.353360 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:19.353328 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:19.860896 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:19.860861 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5854c674c6-czlnf" event={"ID":"6a59376e-4d55-42d0-8c31-0bf26d144a9f","Type":"ContainerStarted","Data":"6acf1f9fbfd66bfba0e237d14275ca4091f20caf91722e02d080d690406613cb"} Jul 02 19:13:19.861292 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:19.860913 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5854c674c6-czlnf" event={"ID":"6a59376e-4d55-42d0-8c31-0bf26d144a9f","Type":"ContainerStarted","Data":"7936e03a5c8f40df8c9e774f73c69384cd5b4760deddc9530fa06d558bd7db02"} Jul 02 19:13:19.879543 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:19.879494 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5854c674c6-czlnf" podStartSLOduration=1.879478794 podStartE2EDuration="1.879478794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:13:19.877861174 +0000 UTC m=+479.804869887" watchObservedRunningTime="2026-07-02 19:13:19.879478794 +0000 UTC m=+479.806487506" Jul 02 19:13:21.096524 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:21.096489 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7"] Jul 02 19:13:21.100012 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:21.099994 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7" Jul 02 19:13:21.102590 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:21.102564 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2chrd\"" Jul 02 19:13:21.102722 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:21.102665 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 19:13:21.102839 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:21.102826 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 19:13:21.107973 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:21.107952 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7"] Jul 02 19:13:21.164723 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:21.164689 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cfcc67b-796c-4fa4-9d84-ad9c8daecd28-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7\" (UID: \"4cfcc67b-796c-4fa4-9d84-ad9c8daecd28\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7" Jul 02 19:13:21.164879 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:21.164743 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cfcc67b-796c-4fa4-9d84-ad9c8daecd28-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7\" (UID: \"4cfcc67b-796c-4fa4-9d84-ad9c8daecd28\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7" Jul 02 19:13:21.164879 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:21.164777 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpwvm\" (UniqueName: \"kubernetes.io/projected/4cfcc67b-796c-4fa4-9d84-ad9c8daecd28-kube-api-access-tpwvm\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7\" (UID: \"4cfcc67b-796c-4fa4-9d84-ad9c8daecd28\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7" Jul 02 19:13:21.265634 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:21.265595 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cfcc67b-796c-4fa4-9d84-ad9c8daecd28-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7\" (UID: \"4cfcc67b-796c-4fa4-9d84-ad9c8daecd28\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7" Jul 02 19:13:21.265835 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:21.265649 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cfcc67b-796c-4fa4-9d84-ad9c8daecd28-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7\" (UID: \"4cfcc67b-796c-4fa4-9d84-ad9c8daecd28\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7" Jul 02 19:13:21.265835 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:21.265719 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpwvm\" (UniqueName: \"kubernetes.io/projected/4cfcc67b-796c-4fa4-9d84-ad9c8daecd28-kube-api-access-tpwvm\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7\" (UID: \"4cfcc67b-796c-4fa4-9d84-ad9c8daecd28\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7" Jul 02 19:13:21.266073 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:21.266045 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cfcc67b-796c-4fa4-9d84-ad9c8daecd28-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7\" (UID: \"4cfcc67b-796c-4fa4-9d84-ad9c8daecd28\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7" Jul 02 19:13:21.266136 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:21.266059 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cfcc67b-796c-4fa4-9d84-ad9c8daecd28-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7\" (UID: \"4cfcc67b-796c-4fa4-9d84-ad9c8daecd28\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7" Jul 02 19:13:21.274514 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:21.274490 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpwvm\" (UniqueName: \"kubernetes.io/projected/4cfcc67b-796c-4fa4-9d84-ad9c8daecd28-kube-api-access-tpwvm\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7\" (UID: \"4cfcc67b-796c-4fa4-9d84-ad9c8daecd28\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7" Jul 02 19:13:21.410226 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:21.410130 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7" Jul 02 19:13:21.538965 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:21.538940 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7"] Jul 02 19:13:21.541287 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:13:21.541263 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cfcc67b_796c_4fa4_9d84_ad9c8daecd28.slice/crio-c26a67d8dfbf5e6aeb36975a6b99f2df0940d665a99e403f2f4a3c3c50c6afdf WatchSource:0}: Error finding container c26a67d8dfbf5e6aeb36975a6b99f2df0940d665a99e403f2f4a3c3c50c6afdf: Status 404 returned error can't find the container with id c26a67d8dfbf5e6aeb36975a6b99f2df0940d665a99e403f2f4a3c3c50c6afdf Jul 02 19:13:21.876879 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:21.876848 2566 generic.go:356] "Generic (PLEG): container finished" podID="4cfcc67b-796c-4fa4-9d84-ad9c8daecd28" containerID="ffcc5f3de0f722323f0e18e5be1d391c3e4a08babd27b798997aff298664b130" exitCode=0 Jul 02 19:13:21.877051 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:21.876884 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7" event={"ID":"4cfcc67b-796c-4fa4-9d84-ad9c8daecd28","Type":"ContainerDied","Data":"ffcc5f3de0f722323f0e18e5be1d391c3e4a08babd27b798997aff298664b130"} Jul 02 19:13:21.877051 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:21.876920 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7" event={"ID":"4cfcc67b-796c-4fa4-9d84-ad9c8daecd28","Type":"ContainerStarted","Data":"c26a67d8dfbf5e6aeb36975a6b99f2df0940d665a99e403f2f4a3c3c50c6afdf"} Jul 02 19:13:22.883657 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:22.883572 2566 generic.go:356] "Generic (PLEG): container finished" podID="4cfcc67b-796c-4fa4-9d84-ad9c8daecd28" containerID="f403970737bc4a276992628f0d4feeabea72320a02eaec0bee78c4175fab4b3a" exitCode=0 Jul 02 19:13:22.883657 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:22.883611 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7" event={"ID":"4cfcc67b-796c-4fa4-9d84-ad9c8daecd28","Type":"ContainerDied","Data":"f403970737bc4a276992628f0d4feeabea72320a02eaec0bee78c4175fab4b3a"} Jul 02 19:13:23.891293 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:23.891261 2566 generic.go:356] "Generic (PLEG): container finished" podID="4cfcc67b-796c-4fa4-9d84-ad9c8daecd28" containerID="df15c0ee4f8c9446b244336c3159c9e06e4820ba53b778903799b07bd7dd0c77" exitCode=0 Jul 02 19:13:23.891665 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:23.891327 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7" event={"ID":"4cfcc67b-796c-4fa4-9d84-ad9c8daecd28","Type":"ContainerDied","Data":"df15c0ee4f8c9446b244336c3159c9e06e4820ba53b778903799b07bd7dd0c77"} Jul 02 19:13:25.024337 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:25.024312 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7" Jul 02 19:13:25.099356 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:25.099325 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cfcc67b-796c-4fa4-9d84-ad9c8daecd28-bundle\") pod \"4cfcc67b-796c-4fa4-9d84-ad9c8daecd28\" (UID: \"4cfcc67b-796c-4fa4-9d84-ad9c8daecd28\") " Jul 02 19:13:25.099540 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:25.099368 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpwvm\" (UniqueName: \"kubernetes.io/projected/4cfcc67b-796c-4fa4-9d84-ad9c8daecd28-kube-api-access-tpwvm\") pod \"4cfcc67b-796c-4fa4-9d84-ad9c8daecd28\" (UID: \"4cfcc67b-796c-4fa4-9d84-ad9c8daecd28\") " Jul 02 19:13:25.099540 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:25.099422 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cfcc67b-796c-4fa4-9d84-ad9c8daecd28-util\") pod \"4cfcc67b-796c-4fa4-9d84-ad9c8daecd28\" (UID: \"4cfcc67b-796c-4fa4-9d84-ad9c8daecd28\") " Jul 02 19:13:25.099871 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:25.099837 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cfcc67b-796c-4fa4-9d84-ad9c8daecd28-bundle" (OuterVolumeSpecName: "bundle") pod "4cfcc67b-796c-4fa4-9d84-ad9c8daecd28" (UID: "4cfcc67b-796c-4fa4-9d84-ad9c8daecd28"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:13:25.101636 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:25.101613 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cfcc67b-796c-4fa4-9d84-ad9c8daecd28-kube-api-access-tpwvm" (OuterVolumeSpecName: "kube-api-access-tpwvm") pod "4cfcc67b-796c-4fa4-9d84-ad9c8daecd28" (UID: "4cfcc67b-796c-4fa4-9d84-ad9c8daecd28"). InnerVolumeSpecName "kube-api-access-tpwvm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:13:25.104775 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:25.104748 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cfcc67b-796c-4fa4-9d84-ad9c8daecd28-util" (OuterVolumeSpecName: "util") pod "4cfcc67b-796c-4fa4-9d84-ad9c8daecd28" (UID: "4cfcc67b-796c-4fa4-9d84-ad9c8daecd28"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:13:25.199996 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:25.199921 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4cfcc67b-796c-4fa4-9d84-ad9c8daecd28-bundle\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:13:25.199996 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:25.199944 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpwvm\" (UniqueName: \"kubernetes.io/projected/4cfcc67b-796c-4fa4-9d84-ad9c8daecd28-kube-api-access-tpwvm\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:13:25.199996 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:25.199954 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4cfcc67b-796c-4fa4-9d84-ad9c8daecd28-util\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:13:25.902273 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:25.902233 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7" event={"ID":"4cfcc67b-796c-4fa4-9d84-ad9c8daecd28","Type":"ContainerDied","Data":"c26a67d8dfbf5e6aeb36975a6b99f2df0940d665a99e403f2f4a3c3c50c6afdf"} Jul 02 19:13:25.902448 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:25.902289 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c26a67d8dfbf5e6aeb36975a6b99f2df0940d665a99e403f2f4a3c3c50c6afdf" Jul 02 19:13:25.902448 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:25.902250 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7" Jul 02 19:13:26.822796 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:26.822760 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:28.795937 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:28.795901 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:28.796311 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:28.795958 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:28.800498 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:28.800473 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:28.919650 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:28.919623 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5854c674c6-czlnf" Jul 02 19:13:28.993144 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:28.993115 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c57cb8c5-hkmfc"] Jul 02 19:13:29.102304 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:29.102205 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:29.235175 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:29.235142 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:29.330472 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:29.330442 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:30.225646 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:30.225606 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:30.665922 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:30.665828 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:37.018988 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:37.018949 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:37.822425 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:37.822382 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:41.901908 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:41.901865 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:42.099121 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:42.099078 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:42.256223 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:42.256189 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:44.286278 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:44.286245 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:44.601421 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:44.601340 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:45.238596 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:45.238564 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:46.659577 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:46.659545 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:47.857076 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:47.857042 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:47.955455 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:47.955422 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:52.217889 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:52.217851 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:52.319161 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:52.319122 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:13:54.014432 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.014394 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-c57cb8c5-hkmfc" podUID="eb7a8d73-8799-4204-bb9d-f9d730abf9a9" containerName="console" containerID="cri-o://dddc262194707b90a868db82f3e7c781c485f3d61ecfb38757dce09aa12aa31e" gracePeriod=15 Jul 02 19:13:54.263453 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.263429 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:13:54.357750 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.357647 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-service-ca\") pod \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " Jul 02 19:13:54.357750 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.357738 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-console-config\") pod \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " Jul 02 19:13:54.357992 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.357768 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfz9r\" (UniqueName: \"kubernetes.io/projected/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-kube-api-access-vfz9r\") pod \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " Jul 02 19:13:54.357992 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.357811 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-oauth-serving-cert\") pod \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " Jul 02 19:13:54.358150 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.358107 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-service-ca" (OuterVolumeSpecName: "service-ca") pod "eb7a8d73-8799-4204-bb9d-f9d730abf9a9" (UID: "eb7a8d73-8799-4204-bb9d-f9d730abf9a9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:13:54.358281 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.358166 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-console-serving-cert\") pod \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " Jul 02 19:13:54.358281 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.358176 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "eb7a8d73-8799-4204-bb9d-f9d730abf9a9" (UID: "eb7a8d73-8799-4204-bb9d-f9d730abf9a9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:13:54.358281 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.358187 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-console-config" (OuterVolumeSpecName: "console-config") pod "eb7a8d73-8799-4204-bb9d-f9d730abf9a9" (UID: "eb7a8d73-8799-4204-bb9d-f9d730abf9a9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:13:54.358281 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.358239 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-console-oauth-config\") pod \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " Jul 02 19:13:54.358281 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.358265 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-trusted-ca-bundle\") pod \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\" (UID: \"eb7a8d73-8799-4204-bb9d-f9d730abf9a9\") " Jul 02 19:13:54.358589 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.358536 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-oauth-serving-cert\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:13:54.358589 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.358556 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-service-ca\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:13:54.358589 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.358569 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-console-config\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:13:54.358808 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.358711 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "eb7a8d73-8799-4204-bb9d-f9d730abf9a9" (UID: "eb7a8d73-8799-4204-bb9d-f9d730abf9a9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 19:13:54.360184 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.360155 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "eb7a8d73-8799-4204-bb9d-f9d730abf9a9" (UID: "eb7a8d73-8799-4204-bb9d-f9d730abf9a9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:13:54.360268 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.360195 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-kube-api-access-vfz9r" (OuterVolumeSpecName: "kube-api-access-vfz9r") pod "eb7a8d73-8799-4204-bb9d-f9d730abf9a9" (UID: "eb7a8d73-8799-4204-bb9d-f9d730abf9a9"). InnerVolumeSpecName "kube-api-access-vfz9r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:13:54.360307 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.360289 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "eb7a8d73-8799-4204-bb9d-f9d730abf9a9" (UID: "eb7a8d73-8799-4204-bb9d-f9d730abf9a9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:13:54.459549 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.459514 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-console-serving-cert\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:13:54.459549 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.459546 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-console-oauth-config\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:13:54.459549 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.459556 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-trusted-ca-bundle\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:13:54.459822 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:54.459566 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfz9r\" (UniqueName: \"kubernetes.io/projected/eb7a8d73-8799-4204-bb9d-f9d730abf9a9-kube-api-access-vfz9r\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:13:55.038709 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.038649 2566 generic.go:356] "Generic (PLEG): container finished" podID="eb7a8d73-8799-4204-bb9d-f9d730abf9a9" containerID="dddc262194707b90a868db82f3e7c781c485f3d61ecfb38757dce09aa12aa31e" exitCode=0 Jul 02 19:13:55.039110 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.038709 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c57cb8c5-hkmfc" event={"ID":"eb7a8d73-8799-4204-bb9d-f9d730abf9a9","Type":"ContainerDied","Data":"dddc262194707b90a868db82f3e7c781c485f3d61ecfb38757dce09aa12aa31e"} Jul 02 19:13:55.039110 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.038768 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c57cb8c5-hkmfc" event={"ID":"eb7a8d73-8799-4204-bb9d-f9d730abf9a9","Type":"ContainerDied","Data":"7bc46b58702338452ea62c24cd6691e1a707b5497dfe5dca2f7fff389cd923b9"} Jul 02 19:13:55.039110 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.038791 2566 scope.go:117] "RemoveContainer" containerID="dddc262194707b90a868db82f3e7c781c485f3d61ecfb38757dce09aa12aa31e" Jul 02 19:13:55.039110 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.038721 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c57cb8c5-hkmfc" Jul 02 19:13:55.047537 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.047520 2566 scope.go:117] "RemoveContainer" containerID="dddc262194707b90a868db82f3e7c781c485f3d61ecfb38757dce09aa12aa31e" Jul 02 19:13:55.047898 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:13:55.047871 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dddc262194707b90a868db82f3e7c781c485f3d61ecfb38757dce09aa12aa31e\": container with ID starting with dddc262194707b90a868db82f3e7c781c485f3d61ecfb38757dce09aa12aa31e not found: ID does not exist" containerID="dddc262194707b90a868db82f3e7c781c485f3d61ecfb38757dce09aa12aa31e" Jul 02 19:13:55.047985 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.047906 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dddc262194707b90a868db82f3e7c781c485f3d61ecfb38757dce09aa12aa31e"} err="failed to get container status \"dddc262194707b90a868db82f3e7c781c485f3d61ecfb38757dce09aa12aa31e\": rpc error: code = NotFound desc = could not find container \"dddc262194707b90a868db82f3e7c781c485f3d61ecfb38757dce09aa12aa31e\": container with ID starting with dddc262194707b90a868db82f3e7c781c485f3d61ecfb38757dce09aa12aa31e not found: ID does not exist" Jul 02 19:13:55.081755 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.081723 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c57cb8c5-hkmfc"] Jul 02 19:13:55.090895 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.089149 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c57cb8c5-hkmfc"] Jul 02 19:13:55.269278 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.269242 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv"] Jul 02 19:13:55.269597 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.269583 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cfcc67b-796c-4fa4-9d84-ad9c8daecd28" containerName="extract" Jul 02 19:13:55.269639 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.269600 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cfcc67b-796c-4fa4-9d84-ad9c8daecd28" containerName="extract" Jul 02 19:13:55.269639 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.269614 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb7a8d73-8799-4204-bb9d-f9d730abf9a9" containerName="console" Jul 02 19:13:55.269639 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.269619 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb7a8d73-8799-4204-bb9d-f9d730abf9a9" containerName="console" Jul 02 19:13:55.269639 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.269636 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cfcc67b-796c-4fa4-9d84-ad9c8daecd28" containerName="util" Jul 02 19:13:55.269797 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.269641 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cfcc67b-796c-4fa4-9d84-ad9c8daecd28" containerName="util" Jul 02 19:13:55.269797 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.269653 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cfcc67b-796c-4fa4-9d84-ad9c8daecd28" containerName="pull" Jul 02 19:13:55.269797 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.269662 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cfcc67b-796c-4fa4-9d84-ad9c8daecd28" containerName="pull" Jul 02 19:13:55.269797 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.269730 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cfcc67b-796c-4fa4-9d84-ad9c8daecd28" containerName="extract" Jul 02 19:13:55.269797 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.269737 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb7a8d73-8799-4204-bb9d-f9d730abf9a9" containerName="console" Jul 02 19:13:55.274026 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.274006 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv" Jul 02 19:13:55.276688 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.276643 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 19:13:55.276688 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.276647 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8jxtl\"" Jul 02 19:13:55.276861 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.276688 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 19:13:55.286844 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.286822 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv"] Jul 02 19:13:55.366820 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.366782 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdch9\" (UniqueName: \"kubernetes.io/projected/f9c6e38e-b0c3-4446-b432-60f9a3a6685b-kube-api-access-tdch9\") pod \"kuadrant-operator-controller-manager-86878c769f-54jgv\" (UID: \"f9c6e38e-b0c3-4446-b432-60f9a3a6685b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv" Jul 02 19:13:55.367023 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.366839 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9c6e38e-b0c3-4446-b432-60f9a3a6685b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-54jgv\" (UID: \"f9c6e38e-b0c3-4446-b432-60f9a3a6685b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv" Jul 02 19:13:55.467604 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.467564 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdch9\" (UniqueName: \"kubernetes.io/projected/f9c6e38e-b0c3-4446-b432-60f9a3a6685b-kube-api-access-tdch9\") pod \"kuadrant-operator-controller-manager-86878c769f-54jgv\" (UID: \"f9c6e38e-b0c3-4446-b432-60f9a3a6685b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv" Jul 02 19:13:55.467807 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.467615 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9c6e38e-b0c3-4446-b432-60f9a3a6685b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-54jgv\" (UID: \"f9c6e38e-b0c3-4446-b432-60f9a3a6685b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv" Jul 02 19:13:55.468094 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.468075 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9c6e38e-b0c3-4446-b432-60f9a3a6685b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-54jgv\" (UID: \"f9c6e38e-b0c3-4446-b432-60f9a3a6685b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv" Jul 02 19:13:55.478834 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.478807 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdch9\" (UniqueName: \"kubernetes.io/projected/f9c6e38e-b0c3-4446-b432-60f9a3a6685b-kube-api-access-tdch9\") pod \"kuadrant-operator-controller-manager-86878c769f-54jgv\" (UID: \"f9c6e38e-b0c3-4446-b432-60f9a3a6685b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv" Jul 02 19:13:55.583646 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.583556 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv" Jul 02 19:13:55.717727 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:55.717690 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv"] Jul 02 19:13:55.720980 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:13:55.720949 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9c6e38e_b0c3_4446_b432_60f9a3a6685b.slice/crio-ed61a0b0cc4b1ab6d6f51e95d78abc9f3de72d9198e73624d783e64f9bc6da2a WatchSource:0}: Error finding container ed61a0b0cc4b1ab6d6f51e95d78abc9f3de72d9198e73624d783e64f9bc6da2a: Status 404 returned error can't find the container with id ed61a0b0cc4b1ab6d6f51e95d78abc9f3de72d9198e73624d783e64f9bc6da2a Jul 02 19:13:56.045196 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:56.045156 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv" event={"ID":"f9c6e38e-b0c3-4446-b432-60f9a3a6685b","Type":"ContainerStarted","Data":"ed61a0b0cc4b1ab6d6f51e95d78abc9f3de72d9198e73624d783e64f9bc6da2a"} Jul 02 19:13:56.844669 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:13:56.844633 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb7a8d73-8799-4204-bb9d-f9d730abf9a9" path="/var/lib/kubelet/pods/eb7a8d73-8799-4204-bb9d-f9d730abf9a9/volumes" Jul 02 19:14:01.636900 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:01.636868 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:01.707323 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:01.707251 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:02.078898 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:02.078858 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv" event={"ID":"f9c6e38e-b0c3-4446-b432-60f9a3a6685b","Type":"ContainerStarted","Data":"cac054e4f64cf88dc0b825ad5dee06dc59f239ca742e7ae1eb87dc5e68377916"} Jul 02 19:14:02.079082 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:02.079007 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv" Jul 02 19:14:02.100506 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:02.100460 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv" podStartSLOduration=1.4277666390000001 podStartE2EDuration="7.100443914s" totalImagesPullingTime="5.672677275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:13:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:14:02.097921409 +0000 UTC m=+522.024930124" watchObservedRunningTime="2026-07-02 19:14:02.100443914 +0000 UTC m=+522.027452625" Jul 02 19:14:02.218593 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:02.218558 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:02.320060 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:02.320025 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:06.042263 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:06.042230 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:06.145579 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:06.145551 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:07.388209 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:07.388176 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:10.226979 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:10.226947 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:10.328736 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:10.328702 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:13.086210 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:13.086130 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv" Jul 02 19:14:13.831080 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:13.831045 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:13.932531 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:13.932496 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:16.521600 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:16.521566 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:16.622573 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:16.622538 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:22.455753 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:22.455720 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:22.560811 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:22.560773 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:25.456955 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:25.456917 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:25.557601 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:25.557566 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:28.058789 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:28.058752 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:28.175118 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:28.175083 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:29.493666 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:29.493624 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:29.678892 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:29.678854 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:29.979863 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:29.979837 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:31.573877 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:31.573841 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:31.675414 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:31.675382 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:31.743651 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:31.743621 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:31.873962 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:31.873873 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:31.966250 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:31.966220 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:32.383585 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:32.383554 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-cfdcp"] Jul 02 19:14:32.387644 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:32.387617 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cfdcp" Jul 02 19:14:32.390206 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:32.390181 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2mdbm\"" Jul 02 19:14:32.392581 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:32.392559 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-cfdcp"] Jul 02 19:14:32.474105 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:32.474070 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:32.493219 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:32.493189 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjgvk\" (UniqueName: \"kubernetes.io/projected/17f7b812-e67c-46bf-9fb9-bd7fd01fdb33-kube-api-access-bjgvk\") pod \"authorino-58cd955675-cfdcp\" (UID: \"17f7b812-e67c-46bf-9fb9-bd7fd01fdb33\") " pod="kuadrant-system/authorino-58cd955675-cfdcp" Jul 02 19:14:32.594313 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:32.594272 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjgvk\" (UniqueName: \"kubernetes.io/projected/17f7b812-e67c-46bf-9fb9-bd7fd01fdb33-kube-api-access-bjgvk\") pod \"authorino-58cd955675-cfdcp\" (UID: \"17f7b812-e67c-46bf-9fb9-bd7fd01fdb33\") " pod="kuadrant-system/authorino-58cd955675-cfdcp" Jul 02 19:14:32.600103 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:32.600076 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:32.602357 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:32.602332 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjgvk\" (UniqueName: \"kubernetes.io/projected/17f7b812-e67c-46bf-9fb9-bd7fd01fdb33-kube-api-access-bjgvk\") pod \"authorino-58cd955675-cfdcp\" (UID: \"17f7b812-e67c-46bf-9fb9-bd7fd01fdb33\") " pod="kuadrant-system/authorino-58cd955675-cfdcp" Jul 02 19:14:32.692699 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:32.692601 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-dfg4h"] Jul 02 19:14:32.694975 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:32.694955 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dfg4h" Jul 02 19:14:32.697914 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:32.697893 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cfdcp" Jul 02 19:14:32.700260 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:32.700239 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dfg4h"] Jul 02 19:14:32.796268 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:32.796239 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v25l\" (UniqueName: \"kubernetes.io/projected/fd61dac4-3682-4f1d-8437-28a88e1e732d-kube-api-access-5v25l\") pod \"authorino-75d556bd77-dfg4h\" (UID: \"fd61dac4-3682-4f1d-8437-28a88e1e732d\") " pod="kuadrant-system/authorino-75d556bd77-dfg4h" Jul 02 19:14:32.827833 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:32.827806 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-cfdcp"] Jul 02 19:14:32.830263 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:14:32.830239 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17f7b812_e67c_46bf_9fb9_bd7fd01fdb33.slice/crio-f88724d4d040bb028089d6c8a17755ac9b48c3088c587cadc11415c1524050cb WatchSource:0}: Error finding container f88724d4d040bb028089d6c8a17755ac9b48c3088c587cadc11415c1524050cb: Status 404 returned error can't find the container with id f88724d4d040bb028089d6c8a17755ac9b48c3088c587cadc11415c1524050cb Jul 02 19:14:32.902269 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:32.897211 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v25l\" (UniqueName: \"kubernetes.io/projected/fd61dac4-3682-4f1d-8437-28a88e1e732d-kube-api-access-5v25l\") pod \"authorino-75d556bd77-dfg4h\" (UID: \"fd61dac4-3682-4f1d-8437-28a88e1e732d\") " pod="kuadrant-system/authorino-75d556bd77-dfg4h" Jul 02 19:14:32.905308 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:32.905288 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v25l\" (UniqueName: \"kubernetes.io/projected/fd61dac4-3682-4f1d-8437-28a88e1e732d-kube-api-access-5v25l\") pod \"authorino-75d556bd77-dfg4h\" (UID: \"fd61dac4-3682-4f1d-8437-28a88e1e732d\") " pod="kuadrant-system/authorino-75d556bd77-dfg4h" Jul 02 19:14:33.005617 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:33.005579 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dfg4h" Jul 02 19:14:33.134167 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:33.134138 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dfg4h"] Jul 02 19:14:33.136248 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:14:33.136224 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd61dac4_3682_4f1d_8437_28a88e1e732d.slice/crio-4ba441a13a8b70f83a2f206473e8b46c608387bbc20394d056b0866174b33a74 WatchSource:0}: Error finding container 4ba441a13a8b70f83a2f206473e8b46c608387bbc20394d056b0866174b33a74: Status 404 returned error can't find the container with id 4ba441a13a8b70f83a2f206473e8b46c608387bbc20394d056b0866174b33a74 Jul 02 19:14:33.226120 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:33.226084 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dfg4h" event={"ID":"fd61dac4-3682-4f1d-8437-28a88e1e732d","Type":"ContainerStarted","Data":"4ba441a13a8b70f83a2f206473e8b46c608387bbc20394d056b0866174b33a74"} Jul 02 19:14:33.227246 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:33.227217 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cfdcp" event={"ID":"17f7b812-e67c-46bf-9fb9-bd7fd01fdb33","Type":"ContainerStarted","Data":"f88724d4d040bb028089d6c8a17755ac9b48c3088c587cadc11415c1524050cb"} Jul 02 19:14:33.274455 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:14:33.274370 2566 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: copying system image from manifest list: determining manifest MIME type for docker://quay.io/kuadrant/authorino:v0.24.0: reading manifest sha256:96b1b9737cf5f546d132e45bd04513096c76a5655e151741306e886e598fc999 in quay.io/kuadrant/authorino: received unexpected HTTP status: 502 Bad Gateway; artifact err: provided artifact is a container image" image="quay.io/kuadrant/authorino:v0.24.0" Jul 02 19:14:33.274607 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:14:33.274475 2566 kuberuntime_manager.go:1459] "Unhandled Error" err="container authorino start failed in pod authorino-58cd955675-cfdcp_kuadrant-system(17f7b812-e67c-46bf-9fb9-bd7fd01fdb33): ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: determining manifest MIME type for docker://quay.io/kuadrant/authorino:v0.24.0: reading manifest sha256:96b1b9737cf5f546d132e45bd04513096c76a5655e151741306e886e598fc999 in quay.io/kuadrant/authorino: received unexpected HTTP status: 502 Bad Gateway; artifact err: provided artifact is a container image" logger="UnhandledError" Jul 02 19:14:33.274607 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:14:33.274508 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"authorino\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: copying system image from manifest list: determining manifest MIME type for docker://quay.io/kuadrant/authorino:v0.24.0: reading manifest sha256:96b1b9737cf5f546d132e45bd04513096c76a5655e151741306e886e598fc999 in quay.io/kuadrant/authorino: received unexpected HTTP status: 502 Bad Gateway; artifact err: provided artifact is a container image\"" pod="kuadrant-system/authorino-58cd955675-cfdcp" podUID="17f7b812-e67c-46bf-9fb9-bd7fd01fdb33" Jul 02 19:14:33.478230 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:33.478196 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:33.580742 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:33.580656 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:34.234790 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:14:34.234752 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"authorino\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/kuadrant/authorino:v0.24.0\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: determining manifest MIME type for docker://quay.io/kuadrant/authorino:v0.24.0: reading manifest sha256:96b1b9737cf5f546d132e45bd04513096c76a5655e151741306e886e598fc999 in quay.io/kuadrant/authorino: received unexpected HTTP status: 502 Bad Gateway; artifact err: provided artifact is a container image\"" pod="kuadrant-system/authorino-58cd955675-cfdcp" podUID="17f7b812-e67c-46bf-9fb9-bd7fd01fdb33" Jul 02 19:14:37.250948 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:37.250906 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dfg4h" event={"ID":"fd61dac4-3682-4f1d-8437-28a88e1e732d","Type":"ContainerStarted","Data":"bbe2cced9d76f9c7501da72e5355cf208e4ee5f04d9d2396d86e7868b9235847"} Jul 02 19:14:37.267542 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:37.267494 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-dfg4h" podStartSLOduration=2.038647122 podStartE2EDuration="5.26747989s" totalImagesPullingTime="3.228832768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:14:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:14:37.265388469 +0000 UTC m=+557.192397181" watchObservedRunningTime="2026-07-02 19:14:37.26747989 +0000 UTC m=+557.194488603" Jul 02 19:14:37.301212 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:37.301180 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-cfdcp"] Jul 02 19:14:37.440339 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:37.440317 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cfdcp" Jul 02 19:14:37.543514 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:37.543412 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjgvk\" (UniqueName: \"kubernetes.io/projected/17f7b812-e67c-46bf-9fb9-bd7fd01fdb33-kube-api-access-bjgvk\") pod \"17f7b812-e67c-46bf-9fb9-bd7fd01fdb33\" (UID: \"17f7b812-e67c-46bf-9fb9-bd7fd01fdb33\") " Jul 02 19:14:37.545462 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:37.545436 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17f7b812-e67c-46bf-9fb9-bd7fd01fdb33-kube-api-access-bjgvk" (OuterVolumeSpecName: "kube-api-access-bjgvk") pod "17f7b812-e67c-46bf-9fb9-bd7fd01fdb33" (UID: "17f7b812-e67c-46bf-9fb9-bd7fd01fdb33"). InnerVolumeSpecName "kube-api-access-bjgvk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:14:37.644269 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:37.644233 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjgvk\" (UniqueName: \"kubernetes.io/projected/17f7b812-e67c-46bf-9fb9-bd7fd01fdb33-kube-api-access-bjgvk\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:14:38.256619 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:38.256591 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-cfdcp" Jul 02 19:14:38.257112 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:38.256652 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-cfdcp" event={"ID":"17f7b812-e67c-46bf-9fb9-bd7fd01fdb33","Type":"ContainerDied","Data":"f88724d4d040bb028089d6c8a17755ac9b48c3088c587cadc11415c1524050cb"} Jul 02 19:14:38.294119 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:38.294061 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-cfdcp"] Jul 02 19:14:38.298515 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:38.298485 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-cfdcp"] Jul 02 19:14:38.844272 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:38.844234 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17f7b812-e67c-46bf-9fb9-bd7fd01fdb33" path="/var/lib/kubelet/pods/17f7b812-e67c-46bf-9fb9-bd7fd01fdb33/volumes" Jul 02 19:14:40.867785 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:40.867742 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:40.969780 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:40.969746 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:43.865629 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:43.865594 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:43.965693 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:43.965652 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:14:53.109032 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:14:53.108994 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:15:05.998323 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:05.998290 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:15:06.214744 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.214709 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-686jk"] Jul 02 19:15:06.218179 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.218156 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-686jk" Jul 02 19:15:06.225573 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.225545 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-686jk"] Jul 02 19:15:06.280981 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.280888 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lcrb\" (UniqueName: \"kubernetes.io/projected/53a71c29-ba37-4977-9cff-6cbb53bc8dc9-kube-api-access-4lcrb\") pod \"authorino-5d4b67c446-686jk\" (UID: \"53a71c29-ba37-4977-9cff-6cbb53bc8dc9\") " pod="kuadrant-system/authorino-5d4b67c446-686jk" Jul 02 19:15:06.381443 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.381404 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lcrb\" (UniqueName: \"kubernetes.io/projected/53a71c29-ba37-4977-9cff-6cbb53bc8dc9-kube-api-access-4lcrb\") pod \"authorino-5d4b67c446-686jk\" (UID: \"53a71c29-ba37-4977-9cff-6cbb53bc8dc9\") " pod="kuadrant-system/authorino-5d4b67c446-686jk" Jul 02 19:15:06.390194 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.390165 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lcrb\" (UniqueName: \"kubernetes.io/projected/53a71c29-ba37-4977-9cff-6cbb53bc8dc9-kube-api-access-4lcrb\") pod \"authorino-5d4b67c446-686jk\" (UID: \"53a71c29-ba37-4977-9cff-6cbb53bc8dc9\") " pod="kuadrant-system/authorino-5d4b67c446-686jk" Jul 02 19:15:06.395775 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.395727 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:15:06.446938 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.446901 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-686jk"] Jul 02 19:15:06.447143 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.447130 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-686jk" Jul 02 19:15:06.477738 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.477697 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5f56f5cd88-bgn96"] Jul 02 19:15:06.481276 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.481255 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f56f5cd88-bgn96" Jul 02 19:15:06.489721 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.488384 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5f56f5cd88-bgn96"] Jul 02 19:15:06.579622 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.579593 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-686jk"] Jul 02 19:15:06.581810 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:15:06.581783 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53a71c29_ba37_4977_9cff_6cbb53bc8dc9.slice/crio-dcb47db8bdb2fd30f839a87542acaeb6d6c4f4ee54538d67b2b509aa39ad6b47 WatchSource:0}: Error finding container dcb47db8bdb2fd30f839a87542acaeb6d6c4f4ee54538d67b2b509aa39ad6b47: Status 404 returned error can't find the container with id dcb47db8bdb2fd30f839a87542acaeb6d6c4f4ee54538d67b2b509aa39ad6b47 Jul 02 19:15:06.582751 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.582728 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh6hb\" (UniqueName: \"kubernetes.io/projected/20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96-kube-api-access-zh6hb\") pod \"authorino-5f56f5cd88-bgn96\" (UID: \"20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96\") " pod="kuadrant-system/authorino-5f56f5cd88-bgn96" Jul 02 19:15:06.683359 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.683320 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh6hb\" (UniqueName: \"kubernetes.io/projected/20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96-kube-api-access-zh6hb\") pod \"authorino-5f56f5cd88-bgn96\" (UID: \"20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96\") " pod="kuadrant-system/authorino-5f56f5cd88-bgn96" Jul 02 19:15:06.691755 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.691723 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh6hb\" (UniqueName: \"kubernetes.io/projected/20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96-kube-api-access-zh6hb\") pod \"authorino-5f56f5cd88-bgn96\" (UID: \"20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96\") " pod="kuadrant-system/authorino-5f56f5cd88-bgn96" Jul 02 19:15:06.736811 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.736775 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5f56f5cd88-bgn96"] Jul 02 19:15:06.737042 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.737029 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f56f5cd88-bgn96" Jul 02 19:15:06.763998 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.763958 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6bd9c7ff4d-ktx4z"] Jul 02 19:15:06.767439 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.767414 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bd9c7ff4d-ktx4z" Jul 02 19:15:06.769612 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.769591 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 02 19:15:06.775178 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.775143 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6bd9c7ff4d-ktx4z"] Jul 02 19:15:06.874433 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.874405 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5f56f5cd88-bgn96"] Jul 02 19:15:06.876523 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:15:06.876492 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20efc7c0_cc1d_4744_9ed4_c7fcda8a3f96.slice/crio-a8eeff9e267cbf63e5f6a0ac4d4ddbc64b3072581bfebab6f662389c4b5c0bdb WatchSource:0}: Error finding container a8eeff9e267cbf63e5f6a0ac4d4ddbc64b3072581bfebab6f662389c4b5c0bdb: Status 404 returned error can't find the container with id a8eeff9e267cbf63e5f6a0ac4d4ddbc64b3072581bfebab6f662389c4b5c0bdb Jul 02 19:15:06.884772 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.884748 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk9f6\" (UniqueName: \"kubernetes.io/projected/d93829de-3bcf-48ee-96af-39473b57055a-kube-api-access-rk9f6\") pod \"authorino-6bd9c7ff4d-ktx4z\" (UID: \"d93829de-3bcf-48ee-96af-39473b57055a\") " pod="kuadrant-system/authorino-6bd9c7ff4d-ktx4z" Jul 02 19:15:06.884876 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.884859 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d93829de-3bcf-48ee-96af-39473b57055a-tls-cert\") pod \"authorino-6bd9c7ff4d-ktx4z\" (UID: \"d93829de-3bcf-48ee-96af-39473b57055a\") " pod="kuadrant-system/authorino-6bd9c7ff4d-ktx4z" Jul 02 19:15:06.986063 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.986031 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d93829de-3bcf-48ee-96af-39473b57055a-tls-cert\") pod \"authorino-6bd9c7ff4d-ktx4z\" (UID: \"d93829de-3bcf-48ee-96af-39473b57055a\") " pod="kuadrant-system/authorino-6bd9c7ff4d-ktx4z" Jul 02 19:15:06.986234 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.986075 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk9f6\" (UniqueName: \"kubernetes.io/projected/d93829de-3bcf-48ee-96af-39473b57055a-kube-api-access-rk9f6\") pod \"authorino-6bd9c7ff4d-ktx4z\" (UID: \"d93829de-3bcf-48ee-96af-39473b57055a\") " pod="kuadrant-system/authorino-6bd9c7ff4d-ktx4z" Jul 02 19:15:06.988704 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.988650 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d93829de-3bcf-48ee-96af-39473b57055a-tls-cert\") pod \"authorino-6bd9c7ff4d-ktx4z\" (UID: \"d93829de-3bcf-48ee-96af-39473b57055a\") " pod="kuadrant-system/authorino-6bd9c7ff4d-ktx4z" Jul 02 19:15:06.993620 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:06.993598 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk9f6\" (UniqueName: \"kubernetes.io/projected/d93829de-3bcf-48ee-96af-39473b57055a-kube-api-access-rk9f6\") pod \"authorino-6bd9c7ff4d-ktx4z\" (UID: \"d93829de-3bcf-48ee-96af-39473b57055a\") " pod="kuadrant-system/authorino-6bd9c7ff4d-ktx4z" Jul 02 19:15:07.080061 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:07.080027 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bd9c7ff4d-ktx4z" Jul 02 19:15:07.251790 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:07.251762 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6bd9c7ff4d-ktx4z"] Jul 02 19:15:07.253473 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:15:07.253444 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd93829de_3bcf_48ee_96af_39473b57055a.slice/crio-8077c88f674a229e97ae383306c4a70d870d52893a565707caa1135df9fe6c82 WatchSource:0}: Error finding container 8077c88f674a229e97ae383306c4a70d870d52893a565707caa1135df9fe6c82: Status 404 returned error can't find the container with id 8077c88f674a229e97ae383306c4a70d870d52893a565707caa1135df9fe6c82 Jul 02 19:15:07.394787 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:07.394746 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bd9c7ff4d-ktx4z" event={"ID":"d93829de-3bcf-48ee-96af-39473b57055a","Type":"ContainerStarted","Data":"8077c88f674a229e97ae383306c4a70d870d52893a565707caa1135df9fe6c82"} Jul 02 19:15:07.396560 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:07.396525 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5f56f5cd88-bgn96" event={"ID":"20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96","Type":"ContainerStarted","Data":"2eee085f40c48dc26221f004fe4adccde0f92cf19ce4cde70aa6d3d6055cf40a"} Jul 02 19:15:07.397000 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:07.396579 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5f56f5cd88-bgn96" event={"ID":"20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96","Type":"ContainerStarted","Data":"a8eeff9e267cbf63e5f6a0ac4d4ddbc64b3072581bfebab6f662389c4b5c0bdb"} Jul 02 19:15:07.397000 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:07.396591 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5f56f5cd88-bgn96" podUID="20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96" containerName="authorino" containerID="cri-o://2eee085f40c48dc26221f004fe4adccde0f92cf19ce4cde70aa6d3d6055cf40a" gracePeriod=30 Jul 02 19:15:07.398897 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:07.398596 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-686jk" event={"ID":"53a71c29-ba37-4977-9cff-6cbb53bc8dc9","Type":"ContainerStarted","Data":"d66e766d902929cf99c8a1469808138edd2f0f65d569a6a8e26b9e05065c28e2"} Jul 02 19:15:07.398897 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:07.398634 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-686jk" event={"ID":"53a71c29-ba37-4977-9cff-6cbb53bc8dc9","Type":"ContainerStarted","Data":"dcb47db8bdb2fd30f839a87542acaeb6d6c4f4ee54538d67b2b509aa39ad6b47"} Jul 02 19:15:07.398897 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:07.398768 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-686jk" podUID="53a71c29-ba37-4977-9cff-6cbb53bc8dc9" containerName="authorino" containerID="cri-o://d66e766d902929cf99c8a1469808138edd2f0f65d569a6a8e26b9e05065c28e2" gracePeriod=30 Jul 02 19:15:07.413760 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:07.413650 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5f56f5cd88-bgn96" podStartSLOduration=1.007526982 podStartE2EDuration="1.413637257s" totalImagesPullingTime="406.110275ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:15:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:15:07.411820876 +0000 UTC m=+587.338829582" watchObservedRunningTime="2026-07-02 19:15:07.413637257 +0000 UTC m=+587.340645971" Jul 02 19:15:07.427572 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:07.427532 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-686jk" podStartSLOduration=1.016298205 podStartE2EDuration="1.427518348s" totalImagesPullingTime="411.220143ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:15:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:15:07.426384381 +0000 UTC m=+587.353393094" watchObservedRunningTime="2026-07-02 19:15:07.427518348 +0000 UTC m=+587.354527080" Jul 02 19:15:07.670328 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:07.670297 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-686jk" Jul 02 19:15:07.673154 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:07.673131 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f56f5cd88-bgn96" Jul 02 19:15:07.792479 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:07.792456 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lcrb\" (UniqueName: \"kubernetes.io/projected/53a71c29-ba37-4977-9cff-6cbb53bc8dc9-kube-api-access-4lcrb\") pod \"53a71c29-ba37-4977-9cff-6cbb53bc8dc9\" (UID: \"53a71c29-ba37-4977-9cff-6cbb53bc8dc9\") " Jul 02 19:15:07.792618 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:07.792578 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zh6hb\" (UniqueName: \"kubernetes.io/projected/20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96-kube-api-access-zh6hb\") pod \"20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96\" (UID: \"20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96\") " Jul 02 19:15:07.794306 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:07.794279 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53a71c29-ba37-4977-9cff-6cbb53bc8dc9-kube-api-access-4lcrb" (OuterVolumeSpecName: "kube-api-access-4lcrb") pod "53a71c29-ba37-4977-9cff-6cbb53bc8dc9" (UID: "53a71c29-ba37-4977-9cff-6cbb53bc8dc9"). InnerVolumeSpecName "kube-api-access-4lcrb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:15:07.794386 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:07.794369 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96-kube-api-access-zh6hb" (OuterVolumeSpecName: "kube-api-access-zh6hb") pod "20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96" (UID: "20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96"). InnerVolumeSpecName "kube-api-access-zh6hb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:15:07.893708 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:07.893596 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zh6hb\" (UniqueName: \"kubernetes.io/projected/20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96-kube-api-access-zh6hb\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:15:07.893708 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:07.893624 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lcrb\" (UniqueName: \"kubernetes.io/projected/53a71c29-ba37-4977-9cff-6cbb53bc8dc9-kube-api-access-4lcrb\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:15:08.405650 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.405619 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bd9c7ff4d-ktx4z" event={"ID":"d93829de-3bcf-48ee-96af-39473b57055a","Type":"ContainerStarted","Data":"46561b78b2b15da641560dda38d20dc4159c40353aa1da1afd2635d9b1884b8e"} Jul 02 19:15:08.406909 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.406888 2566 generic.go:356] "Generic (PLEG): container finished" podID="20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96" containerID="2eee085f40c48dc26221f004fe4adccde0f92cf19ce4cde70aa6d3d6055cf40a" exitCode=0 Jul 02 19:15:08.406974 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.406925 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f56f5cd88-bgn96" Jul 02 19:15:08.406974 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.406961 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5f56f5cd88-bgn96" event={"ID":"20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96","Type":"ContainerDied","Data":"2eee085f40c48dc26221f004fe4adccde0f92cf19ce4cde70aa6d3d6055cf40a"} Jul 02 19:15:08.407060 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.406992 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5f56f5cd88-bgn96" event={"ID":"20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96","Type":"ContainerDied","Data":"a8eeff9e267cbf63e5f6a0ac4d4ddbc64b3072581bfebab6f662389c4b5c0bdb"} Jul 02 19:15:08.407060 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.407015 2566 scope.go:117] "RemoveContainer" containerID="2eee085f40c48dc26221f004fe4adccde0f92cf19ce4cde70aa6d3d6055cf40a" Jul 02 19:15:08.408174 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.408150 2566 generic.go:356] "Generic (PLEG): container finished" podID="53a71c29-ba37-4977-9cff-6cbb53bc8dc9" containerID="d66e766d902929cf99c8a1469808138edd2f0f65d569a6a8e26b9e05065c28e2" exitCode=0 Jul 02 19:15:08.408274 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.408180 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-686jk" Jul 02 19:15:08.408274 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.408229 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-686jk" event={"ID":"53a71c29-ba37-4977-9cff-6cbb53bc8dc9","Type":"ContainerDied","Data":"d66e766d902929cf99c8a1469808138edd2f0f65d569a6a8e26b9e05065c28e2"} Jul 02 19:15:08.408274 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.408260 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-686jk" event={"ID":"53a71c29-ba37-4977-9cff-6cbb53bc8dc9","Type":"ContainerDied","Data":"dcb47db8bdb2fd30f839a87542acaeb6d6c4f4ee54538d67b2b509aa39ad6b47"} Jul 02 19:15:08.416425 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.416409 2566 scope.go:117] "RemoveContainer" containerID="2eee085f40c48dc26221f004fe4adccde0f92cf19ce4cde70aa6d3d6055cf40a" Jul 02 19:15:08.416772 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:15:08.416755 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eee085f40c48dc26221f004fe4adccde0f92cf19ce4cde70aa6d3d6055cf40a\": container with ID starting with 2eee085f40c48dc26221f004fe4adccde0f92cf19ce4cde70aa6d3d6055cf40a not found: ID does not exist" containerID="2eee085f40c48dc26221f004fe4adccde0f92cf19ce4cde70aa6d3d6055cf40a" Jul 02 19:15:08.416850 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.416779 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eee085f40c48dc26221f004fe4adccde0f92cf19ce4cde70aa6d3d6055cf40a"} err="failed to get container status \"2eee085f40c48dc26221f004fe4adccde0f92cf19ce4cde70aa6d3d6055cf40a\": rpc error: code = NotFound desc = could not find container \"2eee085f40c48dc26221f004fe4adccde0f92cf19ce4cde70aa6d3d6055cf40a\": container with ID starting with 2eee085f40c48dc26221f004fe4adccde0f92cf19ce4cde70aa6d3d6055cf40a not found: ID does not exist" Jul 02 19:15:08.416850 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.416796 2566 scope.go:117] "RemoveContainer" containerID="d66e766d902929cf99c8a1469808138edd2f0f65d569a6a8e26b9e05065c28e2" Jul 02 19:15:08.428912 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.425347 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6bd9c7ff4d-ktx4z" podStartSLOduration=1.94183331 podStartE2EDuration="2.425326838s" totalImagesPullingTime="483.493528ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:15:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:15:08.421797363 +0000 UTC m=+588.348806076" watchObservedRunningTime="2026-07-02 19:15:08.425326838 +0000 UTC m=+588.352335551" Jul 02 19:15:08.429559 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.429531 2566 scope.go:117] "RemoveContainer" containerID="d66e766d902929cf99c8a1469808138edd2f0f65d569a6a8e26b9e05065c28e2" Jul 02 19:15:08.430208 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:15:08.430170 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d66e766d902929cf99c8a1469808138edd2f0f65d569a6a8e26b9e05065c28e2\": container with ID starting with d66e766d902929cf99c8a1469808138edd2f0f65d569a6a8e26b9e05065c28e2 not found: ID does not exist" containerID="d66e766d902929cf99c8a1469808138edd2f0f65d569a6a8e26b9e05065c28e2" Jul 02 19:15:08.430309 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.430216 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d66e766d902929cf99c8a1469808138edd2f0f65d569a6a8e26b9e05065c28e2"} err="failed to get container status \"d66e766d902929cf99c8a1469808138edd2f0f65d569a6a8e26b9e05065c28e2\": rpc error: code = NotFound desc = could not find container \"d66e766d902929cf99c8a1469808138edd2f0f65d569a6a8e26b9e05065c28e2\": container with ID starting with d66e766d902929cf99c8a1469808138edd2f0f65d569a6a8e26b9e05065c28e2 not found: ID does not exist" Jul 02 19:15:08.440875 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.440849 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-686jk"] Jul 02 19:15:08.447317 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.447290 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-686jk"] Jul 02 19:15:08.454109 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.454088 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dfg4h"] Jul 02 19:15:08.454264 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.454246 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-dfg4h" podUID="fd61dac4-3682-4f1d-8437-28a88e1e732d" containerName="authorino" containerID="cri-o://bbe2cced9d76f9c7501da72e5355cf208e4ee5f04d9d2396d86e7868b9235847" gracePeriod=30 Jul 02 19:15:08.465745 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.465719 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5f56f5cd88-bgn96"] Jul 02 19:15:08.472442 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.472414 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5f56f5cd88-bgn96"] Jul 02 19:15:08.728451 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.728419 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dfg4h" Jul 02 19:15:08.802192 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.802160 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5v25l\" (UniqueName: \"kubernetes.io/projected/fd61dac4-3682-4f1d-8437-28a88e1e732d-kube-api-access-5v25l\") pod \"fd61dac4-3682-4f1d-8437-28a88e1e732d\" (UID: \"fd61dac4-3682-4f1d-8437-28a88e1e732d\") " Jul 02 19:15:08.804205 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.804176 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd61dac4-3682-4f1d-8437-28a88e1e732d-kube-api-access-5v25l" (OuterVolumeSpecName: "kube-api-access-5v25l") pod "fd61dac4-3682-4f1d-8437-28a88e1e732d" (UID: "fd61dac4-3682-4f1d-8437-28a88e1e732d"). InnerVolumeSpecName "kube-api-access-5v25l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:15:08.844234 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.844206 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96" path="/var/lib/kubelet/pods/20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96/volumes" Jul 02 19:15:08.844542 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.844529 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53a71c29-ba37-4977-9cff-6cbb53bc8dc9" path="/var/lib/kubelet/pods/53a71c29-ba37-4977-9cff-6cbb53bc8dc9/volumes" Jul 02 19:15:08.903172 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:08.903141 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5v25l\" (UniqueName: \"kubernetes.io/projected/fd61dac4-3682-4f1d-8437-28a88e1e732d-kube-api-access-5v25l\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:15:09.416531 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:09.416501 2566 generic.go:356] "Generic (PLEG): container finished" podID="fd61dac4-3682-4f1d-8437-28a88e1e732d" containerID="bbe2cced9d76f9c7501da72e5355cf208e4ee5f04d9d2396d86e7868b9235847" exitCode=0 Jul 02 19:15:09.417053 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:09.416544 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-dfg4h" Jul 02 19:15:09.417053 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:09.416603 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dfg4h" event={"ID":"fd61dac4-3682-4f1d-8437-28a88e1e732d","Type":"ContainerDied","Data":"bbe2cced9d76f9c7501da72e5355cf208e4ee5f04d9d2396d86e7868b9235847"} Jul 02 19:15:09.417053 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:09.416646 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-dfg4h" event={"ID":"fd61dac4-3682-4f1d-8437-28a88e1e732d","Type":"ContainerDied","Data":"4ba441a13a8b70f83a2f206473e8b46c608387bbc20394d056b0866174b33a74"} Jul 02 19:15:09.417053 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:09.416667 2566 scope.go:117] "RemoveContainer" containerID="bbe2cced9d76f9c7501da72e5355cf208e4ee5f04d9d2396d86e7868b9235847" Jul 02 19:15:09.424782 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:09.424756 2566 scope.go:117] "RemoveContainer" containerID="bbe2cced9d76f9c7501da72e5355cf208e4ee5f04d9d2396d86e7868b9235847" Jul 02 19:15:09.425020 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:15:09.424999 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbe2cced9d76f9c7501da72e5355cf208e4ee5f04d9d2396d86e7868b9235847\": container with ID starting with bbe2cced9d76f9c7501da72e5355cf208e4ee5f04d9d2396d86e7868b9235847 not found: ID does not exist" containerID="bbe2cced9d76f9c7501da72e5355cf208e4ee5f04d9d2396d86e7868b9235847" Jul 02 19:15:09.425083 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:09.425032 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbe2cced9d76f9c7501da72e5355cf208e4ee5f04d9d2396d86e7868b9235847"} err="failed to get container status \"bbe2cced9d76f9c7501da72e5355cf208e4ee5f04d9d2396d86e7868b9235847\": rpc error: code = NotFound desc = could not find container \"bbe2cced9d76f9c7501da72e5355cf208e4ee5f04d9d2396d86e7868b9235847\": container with ID starting with bbe2cced9d76f9c7501da72e5355cf208e4ee5f04d9d2396d86e7868b9235847 not found: ID does not exist" Jul 02 19:15:09.433228 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:09.433205 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dfg4h"] Jul 02 19:15:09.438807 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:09.438786 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-dfg4h"] Jul 02 19:15:10.843860 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:10.843828 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd61dac4-3682-4f1d-8437-28a88e1e732d" path="/var/lib/kubelet/pods/fd61dac4-3682-4f1d-8437-28a88e1e732d/volumes" Jul 02 19:15:12.791778 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:12.791745 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:15:12.890456 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:12.890425 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:15:12.979828 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:12.979796 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:15:13.098005 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:13.097913 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:15:16.313986 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:16.313943 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:15:16.405557 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:16.405524 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:15:45.835179 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:45.835085 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:15:46.938624 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:46.938586 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:15:56.879353 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:56.879320 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:15:56.979042 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:56.979006 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:15:59.770696 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:15:59.770639 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:16:07.519694 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:07.519646 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:16:13.378967 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.378932 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp"] Jul 02 19:16:13.379328 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.379279 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53a71c29-ba37-4977-9cff-6cbb53bc8dc9" containerName="authorino" Jul 02 19:16:13.379328 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.379291 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="53a71c29-ba37-4977-9cff-6cbb53bc8dc9" containerName="authorino" Jul 02 19:16:13.379328 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.379315 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96" containerName="authorino" Jul 02 19:16:13.379328 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.379323 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96" containerName="authorino" Jul 02 19:16:13.379459 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.379332 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd61dac4-3682-4f1d-8437-28a88e1e732d" containerName="authorino" Jul 02 19:16:13.379459 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.379337 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd61dac4-3682-4f1d-8437-28a88e1e732d" containerName="authorino" Jul 02 19:16:13.379459 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.379403 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="53a71c29-ba37-4977-9cff-6cbb53bc8dc9" containerName="authorino" Jul 02 19:16:13.379459 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.379410 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd61dac4-3682-4f1d-8437-28a88e1e732d" containerName="authorino" Jul 02 19:16:13.379459 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.379416 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="20efc7c0-cc1d-4744-9ed4-c7fcda8a3f96" containerName="authorino" Jul 02 19:16:13.382419 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.382401 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:13.385816 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.385778 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kkw25\"" Jul 02 19:16:13.385816 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.385787 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 02 19:16:13.386030 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.385814 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 02 19:16:13.386030 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.385845 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 02 19:16:13.393495 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.393471 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp"] Jul 02 19:16:13.412729 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.412700 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:16:13.562048 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.562008 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ff052e2-e15b-4e4e-871f-1f4c943d9498-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c72fp\" (UID: \"4ff052e2-e15b-4e4e-871f-1f4c943d9498\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:13.562229 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.562099 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ff052e2-e15b-4e4e-871f-1f4c943d9498-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c72fp\" (UID: \"4ff052e2-e15b-4e4e-871f-1f4c943d9498\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:13.562229 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.562136 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ff052e2-e15b-4e4e-871f-1f4c943d9498-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c72fp\" (UID: \"4ff052e2-e15b-4e4e-871f-1f4c943d9498\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:13.562229 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.562176 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ff052e2-e15b-4e4e-871f-1f4c943d9498-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c72fp\" (UID: \"4ff052e2-e15b-4e4e-871f-1f4c943d9498\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:13.562355 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.562234 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ff052e2-e15b-4e4e-871f-1f4c943d9498-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c72fp\" (UID: \"4ff052e2-e15b-4e4e-871f-1f4c943d9498\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:13.562355 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.562269 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwh7p\" (UniqueName: \"kubernetes.io/projected/4ff052e2-e15b-4e4e-871f-1f4c943d9498-kube-api-access-lwh7p\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c72fp\" (UID: \"4ff052e2-e15b-4e4e-871f-1f4c943d9498\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:13.663233 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.663145 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ff052e2-e15b-4e4e-871f-1f4c943d9498-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c72fp\" (UID: \"4ff052e2-e15b-4e4e-871f-1f4c943d9498\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:13.663233 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.663182 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwh7p\" (UniqueName: \"kubernetes.io/projected/4ff052e2-e15b-4e4e-871f-1f4c943d9498-kube-api-access-lwh7p\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c72fp\" (UID: \"4ff052e2-e15b-4e4e-871f-1f4c943d9498\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:13.663233 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.663215 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ff052e2-e15b-4e4e-871f-1f4c943d9498-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c72fp\" (UID: \"4ff052e2-e15b-4e4e-871f-1f4c943d9498\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:13.663530 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.663251 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ff052e2-e15b-4e4e-871f-1f4c943d9498-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c72fp\" (UID: \"4ff052e2-e15b-4e4e-871f-1f4c943d9498\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:13.663530 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.663270 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ff052e2-e15b-4e4e-871f-1f4c943d9498-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c72fp\" (UID: \"4ff052e2-e15b-4e4e-871f-1f4c943d9498\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:13.663530 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.663293 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ff052e2-e15b-4e4e-871f-1f4c943d9498-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c72fp\" (UID: \"4ff052e2-e15b-4e4e-871f-1f4c943d9498\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:13.663776 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.663603 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ff052e2-e15b-4e4e-871f-1f4c943d9498-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c72fp\" (UID: \"4ff052e2-e15b-4e4e-871f-1f4c943d9498\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:13.663776 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.663634 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ff052e2-e15b-4e4e-871f-1f4c943d9498-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c72fp\" (UID: \"4ff052e2-e15b-4e4e-871f-1f4c943d9498\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:13.663776 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.663659 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ff052e2-e15b-4e4e-871f-1f4c943d9498-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c72fp\" (UID: \"4ff052e2-e15b-4e4e-871f-1f4c943d9498\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:13.665580 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.665559 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ff052e2-e15b-4e4e-871f-1f4c943d9498-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c72fp\" (UID: \"4ff052e2-e15b-4e4e-871f-1f4c943d9498\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:13.665859 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.665831 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ff052e2-e15b-4e4e-871f-1f4c943d9498-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c72fp\" (UID: \"4ff052e2-e15b-4e4e-871f-1f4c943d9498\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:13.671087 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.671065 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwh7p\" (UniqueName: \"kubernetes.io/projected/4ff052e2-e15b-4e4e-871f-1f4c943d9498-kube-api-access-lwh7p\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c72fp\" (UID: \"4ff052e2-e15b-4e4e-871f-1f4c943d9498\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:13.692058 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.692027 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:13.828036 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.827971 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp"] Jul 02 19:16:13.830223 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:16:13.830197 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ff052e2_e15b_4e4e_871f_1f4c943d9498.slice/crio-5df7e29fd3058140ec0d641c03a68488330e800e1cc52ea5160a8f65ac09b8e6 WatchSource:0}: Error finding container 5df7e29fd3058140ec0d641c03a68488330e800e1cc52ea5160a8f65ac09b8e6: Status 404 returned error can't find the container with id 5df7e29fd3058140ec0d641c03a68488330e800e1cc52ea5160a8f65ac09b8e6 Jul 02 19:16:13.831577 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:13.831554 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 19:16:14.728385 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:14.728337 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" event={"ID":"4ff052e2-e15b-4e4e-871f-1f4c943d9498","Type":"ContainerStarted","Data":"5df7e29fd3058140ec0d641c03a68488330e800e1cc52ea5160a8f65ac09b8e6"} Jul 02 19:16:15.734638 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:15.734605 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" event={"ID":"4ff052e2-e15b-4e4e-871f-1f4c943d9498","Type":"ContainerStarted","Data":"593a40c900f9804f49968523c7d679df34ca99401a7a8771f5f932dbec9951ff"} Jul 02 19:16:15.757303 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:15.757251 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" podStartSLOduration=1.467721166 podStartE2EDuration="2.757235054s" totalImagesPullingTime="1.289513888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:16:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:16:15.754254336 +0000 UTC m=+655.681263049" watchObservedRunningTime="2026-07-02 19:16:15.757235054 +0000 UTC m=+655.684243765" Jul 02 19:16:17.902848 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:17.902817 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:16:23.692354 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:23.692306 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:23.692805 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:23.692382 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:23.704958 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:23.704935 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:23.783222 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:23.783191 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c72fp" Jul 02 19:16:34.086961 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.086926 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx"] Jul 02 19:16:34.090433 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.090413 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:34.093527 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.093504 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 02 19:16:34.106113 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.106086 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx"] Jul 02 19:16:34.122690 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.122651 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:16:34.133825 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.133799 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85f4eefb-e21a-4bc5-9d05-532dd0a2328c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx\" (UID: \"85f4eefb-e21a-4bc5-9d05-532dd0a2328c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:34.133955 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.133849 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg62r\" (UniqueName: \"kubernetes.io/projected/85f4eefb-e21a-4bc5-9d05-532dd0a2328c-kube-api-access-dg62r\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx\" (UID: \"85f4eefb-e21a-4bc5-9d05-532dd0a2328c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:34.133955 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.133915 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85f4eefb-e21a-4bc5-9d05-532dd0a2328c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx\" (UID: \"85f4eefb-e21a-4bc5-9d05-532dd0a2328c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:34.133955 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.133946 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85f4eefb-e21a-4bc5-9d05-532dd0a2328c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx\" (UID: \"85f4eefb-e21a-4bc5-9d05-532dd0a2328c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:34.134068 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.134004 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85f4eefb-e21a-4bc5-9d05-532dd0a2328c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx\" (UID: \"85f4eefb-e21a-4bc5-9d05-532dd0a2328c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:34.134068 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.134024 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85f4eefb-e21a-4bc5-9d05-532dd0a2328c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx\" (UID: \"85f4eefb-e21a-4bc5-9d05-532dd0a2328c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:34.234862 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.234824 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85f4eefb-e21a-4bc5-9d05-532dd0a2328c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx\" (UID: \"85f4eefb-e21a-4bc5-9d05-532dd0a2328c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:34.234862 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.234861 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85f4eefb-e21a-4bc5-9d05-532dd0a2328c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx\" (UID: \"85f4eefb-e21a-4bc5-9d05-532dd0a2328c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:34.235108 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.234910 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85f4eefb-e21a-4bc5-9d05-532dd0a2328c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx\" (UID: \"85f4eefb-e21a-4bc5-9d05-532dd0a2328c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:34.235108 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.234951 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg62r\" (UniqueName: \"kubernetes.io/projected/85f4eefb-e21a-4bc5-9d05-532dd0a2328c-kube-api-access-dg62r\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx\" (UID: \"85f4eefb-e21a-4bc5-9d05-532dd0a2328c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:34.235108 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.234981 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85f4eefb-e21a-4bc5-9d05-532dd0a2328c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx\" (UID: \"85f4eefb-e21a-4bc5-9d05-532dd0a2328c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:34.235108 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.235005 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85f4eefb-e21a-4bc5-9d05-532dd0a2328c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx\" (UID: \"85f4eefb-e21a-4bc5-9d05-532dd0a2328c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:34.235369 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.235338 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85f4eefb-e21a-4bc5-9d05-532dd0a2328c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx\" (UID: \"85f4eefb-e21a-4bc5-9d05-532dd0a2328c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:34.235443 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.235380 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85f4eefb-e21a-4bc5-9d05-532dd0a2328c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx\" (UID: \"85f4eefb-e21a-4bc5-9d05-532dd0a2328c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:34.235502 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.235470 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85f4eefb-e21a-4bc5-9d05-532dd0a2328c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx\" (UID: \"85f4eefb-e21a-4bc5-9d05-532dd0a2328c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:34.237218 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.237196 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85f4eefb-e21a-4bc5-9d05-532dd0a2328c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx\" (UID: \"85f4eefb-e21a-4bc5-9d05-532dd0a2328c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:34.237358 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.237339 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85f4eefb-e21a-4bc5-9d05-532dd0a2328c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx\" (UID: \"85f4eefb-e21a-4bc5-9d05-532dd0a2328c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:34.243854 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.243828 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg62r\" (UniqueName: \"kubernetes.io/projected/85f4eefb-e21a-4bc5-9d05-532dd0a2328c-kube-api-access-dg62r\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx\" (UID: \"85f4eefb-e21a-4bc5-9d05-532dd0a2328c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:34.401165 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.401078 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:34.532551 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.532515 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx"] Jul 02 19:16:34.535751 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:16:34.535724 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85f4eefb_e21a_4bc5_9d05_532dd0a2328c.slice/crio-99644d311aa7a7472f53184c6f8d67e1f72ee5a237874161c2d0c2ffb2dedeaa WatchSource:0}: Error finding container 99644d311aa7a7472f53184c6f8d67e1f72ee5a237874161c2d0c2ffb2dedeaa: Status 404 returned error can't find the container with id 99644d311aa7a7472f53184c6f8d67e1f72ee5a237874161c2d0c2ffb2dedeaa Jul 02 19:16:34.822275 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:34.822250 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" event={"ID":"85f4eefb-e21a-4bc5-9d05-532dd0a2328c","Type":"ContainerStarted","Data":"99644d311aa7a7472f53184c6f8d67e1f72ee5a237874161c2d0c2ffb2dedeaa"} Jul 02 19:16:35.829217 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:35.829175 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" event={"ID":"85f4eefb-e21a-4bc5-9d05-532dd0a2328c","Type":"ContainerStarted","Data":"fd95dd2ca964b0a0ec74ca42cb83de8f2a36bfde5f2667816b081066f6123b3d"} Jul 02 19:16:35.848560 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:35.848506 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" podStartSLOduration=1.586297379 podStartE2EDuration="1.8484911s" totalImagesPullingTime="262.193721ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:16:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:16:35.847372151 +0000 UTC m=+675.774380862" watchObservedRunningTime="2026-07-02 19:16:35.8484911 +0000 UTC m=+675.775499811" Jul 02 19:16:44.401652 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:44.401613 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:44.402114 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:44.401696 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:44.414116 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:44.414085 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:44.886035 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:44.886010 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx" Jul 02 19:16:52.919138 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:52.919104 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:16:57.876265 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:57.876220 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9"] Jul 02 19:16:57.882956 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:57.882938 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:16:57.885282 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:57.885262 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 02 19:16:57.888965 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:57.888939 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9"] Jul 02 19:16:57.906180 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:57.906153 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:16:58.051351 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.051308 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c84a8d95-1ddf-4d7e-b875-693224a0e3c0-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9\" (UID: \"c84a8d95-1ddf-4d7e-b875-693224a0e3c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:16:58.051351 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.051354 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c84a8d95-1ddf-4d7e-b875-693224a0e3c0-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9\" (UID: \"c84a8d95-1ddf-4d7e-b875-693224a0e3c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:16:58.051593 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.051412 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c84a8d95-1ddf-4d7e-b875-693224a0e3c0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9\" (UID: \"c84a8d95-1ddf-4d7e-b875-693224a0e3c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:16:58.051593 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.051436 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c84a8d95-1ddf-4d7e-b875-693224a0e3c0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9\" (UID: \"c84a8d95-1ddf-4d7e-b875-693224a0e3c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:16:58.051593 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.051501 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qqsv\" (UniqueName: \"kubernetes.io/projected/c84a8d95-1ddf-4d7e-b875-693224a0e3c0-kube-api-access-6qqsv\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9\" (UID: \"c84a8d95-1ddf-4d7e-b875-693224a0e3c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:16:58.051593 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.051534 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c84a8d95-1ddf-4d7e-b875-693224a0e3c0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9\" (UID: \"c84a8d95-1ddf-4d7e-b875-693224a0e3c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:16:58.152425 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.152323 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c84a8d95-1ddf-4d7e-b875-693224a0e3c0-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9\" (UID: \"c84a8d95-1ddf-4d7e-b875-693224a0e3c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:16:58.152425 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.152373 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c84a8d95-1ddf-4d7e-b875-693224a0e3c0-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9\" (UID: \"c84a8d95-1ddf-4d7e-b875-693224a0e3c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:16:58.152425 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.152411 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c84a8d95-1ddf-4d7e-b875-693224a0e3c0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9\" (UID: \"c84a8d95-1ddf-4d7e-b875-693224a0e3c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:16:58.152698 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.152446 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c84a8d95-1ddf-4d7e-b875-693224a0e3c0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9\" (UID: \"c84a8d95-1ddf-4d7e-b875-693224a0e3c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:16:58.152698 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.152581 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qqsv\" (UniqueName: \"kubernetes.io/projected/c84a8d95-1ddf-4d7e-b875-693224a0e3c0-kube-api-access-6qqsv\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9\" (UID: \"c84a8d95-1ddf-4d7e-b875-693224a0e3c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:16:58.152698 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.152626 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c84a8d95-1ddf-4d7e-b875-693224a0e3c0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9\" (UID: \"c84a8d95-1ddf-4d7e-b875-693224a0e3c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:16:58.152871 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.152849 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c84a8d95-1ddf-4d7e-b875-693224a0e3c0-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9\" (UID: \"c84a8d95-1ddf-4d7e-b875-693224a0e3c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:16:58.152931 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.152895 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c84a8d95-1ddf-4d7e-b875-693224a0e3c0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9\" (UID: \"c84a8d95-1ddf-4d7e-b875-693224a0e3c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:16:58.153077 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.153059 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c84a8d95-1ddf-4d7e-b875-693224a0e3c0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9\" (UID: \"c84a8d95-1ddf-4d7e-b875-693224a0e3c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:16:58.154709 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.154665 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c84a8d95-1ddf-4d7e-b875-693224a0e3c0-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9\" (UID: \"c84a8d95-1ddf-4d7e-b875-693224a0e3c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:16:58.155104 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.155085 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c84a8d95-1ddf-4d7e-b875-693224a0e3c0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9\" (UID: \"c84a8d95-1ddf-4d7e-b875-693224a0e3c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:16:58.160689 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.160661 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qqsv\" (UniqueName: \"kubernetes.io/projected/c84a8d95-1ddf-4d7e-b875-693224a0e3c0-kube-api-access-6qqsv\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9\" (UID: \"c84a8d95-1ddf-4d7e-b875-693224a0e3c0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:16:58.193562 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.193515 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:16:58.330959 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.330933 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9"] Jul 02 19:16:58.333969 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:16:58.333942 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc84a8d95_1ddf_4d7e_b875_693224a0e3c0.slice/crio-813c6451b9426999bef8dae8c1e3d58a9065ee67e98217fd5432515eeaaf8cd4 WatchSource:0}: Error finding container 813c6451b9426999bef8dae8c1e3d58a9065ee67e98217fd5432515eeaaf8cd4: Status 404 returned error can't find the container with id 813c6451b9426999bef8dae8c1e3d58a9065ee67e98217fd5432515eeaaf8cd4 Jul 02 19:16:58.943378 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.943286 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" event={"ID":"c84a8d95-1ddf-4d7e-b875-693224a0e3c0","Type":"ContainerStarted","Data":"073795ff9f968baf59cb86f80f5d184f1dbb59d0574f20471805ecf8dabc96c1"} Jul 02 19:16:58.943378 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.943326 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" event={"ID":"c84a8d95-1ddf-4d7e-b875-693224a0e3c0","Type":"ContainerStarted","Data":"813c6451b9426999bef8dae8c1e3d58a9065ee67e98217fd5432515eeaaf8cd4"} Jul 02 19:16:58.964190 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:16:58.964130 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" podStartSLOduration=1.695387431 podStartE2EDuration="1.9641113s" totalImagesPullingTime="268.723869ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:16:58.962048626 +0000 UTC m=+698.889057340" watchObservedRunningTime="2026-07-02 19:16:58.9641113 +0000 UTC m=+698.891120013" Jul 02 19:17:08.194176 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:08.194120 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:17:08.194176 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:08.194183 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:17:08.209612 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:08.209582 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:17:09.002314 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:09.002286 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9" Jul 02 19:17:45.751633 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:45.751555 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c4bf44fc-h2bfx"] Jul 02 19:17:45.754856 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:45.754838 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c4bf44fc-h2bfx" Jul 02 19:17:45.764581 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:45.764554 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c4bf44fc-h2bfx"] Jul 02 19:17:45.857763 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:45.857722 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0a1736aa-2aed-4664-9891-1ad629369467-tls-cert\") pod \"authorino-59c4bf44fc-h2bfx\" (UID: \"0a1736aa-2aed-4664-9891-1ad629369467\") " pod="kuadrant-system/authorino-59c4bf44fc-h2bfx" Jul 02 19:17:45.857951 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:45.857805 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2p2v\" (UniqueName: \"kubernetes.io/projected/0a1736aa-2aed-4664-9891-1ad629369467-kube-api-access-r2p2v\") pod \"authorino-59c4bf44fc-h2bfx\" (UID: \"0a1736aa-2aed-4664-9891-1ad629369467\") " pod="kuadrant-system/authorino-59c4bf44fc-h2bfx" Jul 02 19:17:45.958289 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:45.958250 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2p2v\" (UniqueName: \"kubernetes.io/projected/0a1736aa-2aed-4664-9891-1ad629369467-kube-api-access-r2p2v\") pod \"authorino-59c4bf44fc-h2bfx\" (UID: \"0a1736aa-2aed-4664-9891-1ad629369467\") " pod="kuadrant-system/authorino-59c4bf44fc-h2bfx" Jul 02 19:17:45.958440 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:45.958318 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0a1736aa-2aed-4664-9891-1ad629369467-tls-cert\") pod \"authorino-59c4bf44fc-h2bfx\" (UID: \"0a1736aa-2aed-4664-9891-1ad629369467\") " pod="kuadrant-system/authorino-59c4bf44fc-h2bfx" Jul 02 19:17:45.960887 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:45.960857 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0a1736aa-2aed-4664-9891-1ad629369467-tls-cert\") pod \"authorino-59c4bf44fc-h2bfx\" (UID: \"0a1736aa-2aed-4664-9891-1ad629369467\") " pod="kuadrant-system/authorino-59c4bf44fc-h2bfx" Jul 02 19:17:45.967565 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:45.967536 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2p2v\" (UniqueName: \"kubernetes.io/projected/0a1736aa-2aed-4664-9891-1ad629369467-kube-api-access-r2p2v\") pod \"authorino-59c4bf44fc-h2bfx\" (UID: \"0a1736aa-2aed-4664-9891-1ad629369467\") " pod="kuadrant-system/authorino-59c4bf44fc-h2bfx" Jul 02 19:17:46.063955 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:46.063870 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c4bf44fc-h2bfx" Jul 02 19:17:46.221177 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:46.221143 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c4bf44fc-h2bfx"] Jul 02 19:17:46.224281 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:17:46.224256 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a1736aa_2aed_4664_9891_1ad629369467.slice/crio-1b6f1aca985dbc2dacbaa7d9bc6208e15ebee51524bcfd0bb31416956fc92bab WatchSource:0}: Error finding container 1b6f1aca985dbc2dacbaa7d9bc6208e15ebee51524bcfd0bb31416956fc92bab: Status 404 returned error can't find the container with id 1b6f1aca985dbc2dacbaa7d9bc6208e15ebee51524bcfd0bb31416956fc92bab Jul 02 19:17:47.184661 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:47.184607 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c4bf44fc-h2bfx" event={"ID":"0a1736aa-2aed-4664-9891-1ad629369467","Type":"ContainerStarted","Data":"e5f2b3d9a2e42dfe3969e6b4c1bad7382c54f58b86edeb5ac99cbe5742e3fa23"} Jul 02 19:17:47.185170 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:47.184695 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c4bf44fc-h2bfx" event={"ID":"0a1736aa-2aed-4664-9891-1ad629369467","Type":"ContainerStarted","Data":"1b6f1aca985dbc2dacbaa7d9bc6208e15ebee51524bcfd0bb31416956fc92bab"} Jul 02 19:17:47.208257 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:47.208192 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c4bf44fc-h2bfx" podStartSLOduration=1.817297363 podStartE2EDuration="2.208170384s" totalImagesPullingTime="390.873021ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:17:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:17:47.205522869 +0000 UTC m=+747.132531582" watchObservedRunningTime="2026-07-02 19:17:47.208170384 +0000 UTC m=+747.135179097" Jul 02 19:17:47.246820 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:47.246787 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6bd9c7ff4d-ktx4z"] Jul 02 19:17:47.247239 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:47.247190 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-6bd9c7ff4d-ktx4z" podUID="d93829de-3bcf-48ee-96af-39473b57055a" containerName="authorino" containerID="cri-o://46561b78b2b15da641560dda38d20dc4159c40353aa1da1afd2635d9b1884b8e" gracePeriod=30 Jul 02 19:17:47.503441 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:47.503405 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bd9c7ff4d-ktx4z" Jul 02 19:17:47.576302 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:47.576256 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rk9f6\" (UniqueName: \"kubernetes.io/projected/d93829de-3bcf-48ee-96af-39473b57055a-kube-api-access-rk9f6\") pod \"d93829de-3bcf-48ee-96af-39473b57055a\" (UID: \"d93829de-3bcf-48ee-96af-39473b57055a\") " Jul 02 19:17:47.576499 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:47.576323 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d93829de-3bcf-48ee-96af-39473b57055a-tls-cert\") pod \"d93829de-3bcf-48ee-96af-39473b57055a\" (UID: \"d93829de-3bcf-48ee-96af-39473b57055a\") " Jul 02 19:17:47.578399 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:47.578365 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d93829de-3bcf-48ee-96af-39473b57055a-kube-api-access-rk9f6" (OuterVolumeSpecName: "kube-api-access-rk9f6") pod "d93829de-3bcf-48ee-96af-39473b57055a" (UID: "d93829de-3bcf-48ee-96af-39473b57055a"). InnerVolumeSpecName "kube-api-access-rk9f6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:17:47.587898 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:47.587871 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d93829de-3bcf-48ee-96af-39473b57055a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d93829de-3bcf-48ee-96af-39473b57055a" (UID: "d93829de-3bcf-48ee-96af-39473b57055a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:17:47.677334 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:47.677301 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rk9f6\" (UniqueName: \"kubernetes.io/projected/d93829de-3bcf-48ee-96af-39473b57055a-kube-api-access-rk9f6\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:17:47.677334 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:47.677326 2566 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d93829de-3bcf-48ee-96af-39473b57055a-tls-cert\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:17:47.779952 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:47.779862 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:17:47.881311 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:47.881280 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:17:48.192215 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:48.192123 2566 generic.go:356] "Generic (PLEG): container finished" podID="d93829de-3bcf-48ee-96af-39473b57055a" containerID="46561b78b2b15da641560dda38d20dc4159c40353aa1da1afd2635d9b1884b8e" exitCode=0 Jul 02 19:17:48.192215 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:48.192170 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bd9c7ff4d-ktx4z" Jul 02 19:17:48.192655 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:48.192210 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bd9c7ff4d-ktx4z" event={"ID":"d93829de-3bcf-48ee-96af-39473b57055a","Type":"ContainerDied","Data":"46561b78b2b15da641560dda38d20dc4159c40353aa1da1afd2635d9b1884b8e"} Jul 02 19:17:48.192655 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:48.192259 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bd9c7ff4d-ktx4z" event={"ID":"d93829de-3bcf-48ee-96af-39473b57055a","Type":"ContainerDied","Data":"8077c88f674a229e97ae383306c4a70d870d52893a565707caa1135df9fe6c82"} Jul 02 19:17:48.192655 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:48.192282 2566 scope.go:117] "RemoveContainer" containerID="46561b78b2b15da641560dda38d20dc4159c40353aa1da1afd2635d9b1884b8e" Jul 02 19:17:48.202898 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:48.202847 2566 scope.go:117] "RemoveContainer" containerID="46561b78b2b15da641560dda38d20dc4159c40353aa1da1afd2635d9b1884b8e" Jul 02 19:17:48.203227 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:17:48.203159 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46561b78b2b15da641560dda38d20dc4159c40353aa1da1afd2635d9b1884b8e\": container with ID starting with 46561b78b2b15da641560dda38d20dc4159c40353aa1da1afd2635d9b1884b8e not found: ID does not exist" containerID="46561b78b2b15da641560dda38d20dc4159c40353aa1da1afd2635d9b1884b8e" Jul 02 19:17:48.203227 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:48.203198 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46561b78b2b15da641560dda38d20dc4159c40353aa1da1afd2635d9b1884b8e"} err="failed to get container status \"46561b78b2b15da641560dda38d20dc4159c40353aa1da1afd2635d9b1884b8e\": rpc error: code = NotFound desc = could not find container \"46561b78b2b15da641560dda38d20dc4159c40353aa1da1afd2635d9b1884b8e\": container with ID starting with 46561b78b2b15da641560dda38d20dc4159c40353aa1da1afd2635d9b1884b8e not found: ID does not exist" Jul 02 19:17:48.219875 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:48.219844 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6bd9c7ff4d-ktx4z"] Jul 02 19:17:48.228556 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:48.228527 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6bd9c7ff4d-ktx4z"] Jul 02 19:17:48.844907 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:48.844874 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d93829de-3bcf-48ee-96af-39473b57055a" path="/var/lib/kubelet/pods/d93829de-3bcf-48ee-96af-39473b57055a/volumes" Jul 02 19:17:48.923247 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:48.923216 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:17:49.021221 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:17:49.021187 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:19:04.595688 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:19:04.595653 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:19:06.645100 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:19:06.645065 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:19:39.697312 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:19:39.697273 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:19:39.798013 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:19:39.797965 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:19:39.947140 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:19:39.947101 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:19:45.781200 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:19:45.781163 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:19:45.853403 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:19:45.853368 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:19:45.946407 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:19:45.946372 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:19:46.043610 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:19:46.043529 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:19:46.163617 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:19:46.163588 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:19:51.964083 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:19:51.964041 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:19:52.032358 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:19:52.032323 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:20:09.979934 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:20:09.979897 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:20:11.416811 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:20:11.416772 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:20:26.956488 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:20:26.956454 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:20:50.170268 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:20:50.170234 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:20:51.506421 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:20:51.506386 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:21:36.484372 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:21:36.484338 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:22:06.302696 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:22:06.302633 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:22:06.403184 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:22:06.403146 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:22:15.950829 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:22:15.950797 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:22:16.052433 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:22:16.052399 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:23:17.458287 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:23:17.458206 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:23:45.591858 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:23:45.591811 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv"] Jul 02 19:23:45.592347 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:23:45.592022 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv" podUID="f9c6e38e-b0c3-4446-b432-60f9a3a6685b" containerName="manager" containerID="cri-o://cac054e4f64cf88dc0b825ad5dee06dc59f239ca742e7ae1eb87dc5e68377916" gracePeriod=10 Jul 02 19:23:48.744003 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:23:48.743981 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv" Jul 02 19:23:48.798458 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:23:48.798364 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9c6e38e-b0c3-4446-b432-60f9a3a6685b-extensions-socket-volume\") pod \"f9c6e38e-b0c3-4446-b432-60f9a3a6685b\" (UID: \"f9c6e38e-b0c3-4446-b432-60f9a3a6685b\") " Jul 02 19:23:48.798458 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:23:48.798426 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdch9\" (UniqueName: \"kubernetes.io/projected/f9c6e38e-b0c3-4446-b432-60f9a3a6685b-kube-api-access-tdch9\") pod \"f9c6e38e-b0c3-4446-b432-60f9a3a6685b\" (UID: \"f9c6e38e-b0c3-4446-b432-60f9a3a6685b\") " Jul 02 19:23:48.798769 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:23:48.798743 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9c6e38e-b0c3-4446-b432-60f9a3a6685b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f9c6e38e-b0c3-4446-b432-60f9a3a6685b" (UID: "f9c6e38e-b0c3-4446-b432-60f9a3a6685b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:23:48.800570 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:23:48.800548 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9c6e38e-b0c3-4446-b432-60f9a3a6685b-kube-api-access-tdch9" (OuterVolumeSpecName: "kube-api-access-tdch9") pod "f9c6e38e-b0c3-4446-b432-60f9a3a6685b" (UID: "f9c6e38e-b0c3-4446-b432-60f9a3a6685b"). InnerVolumeSpecName "kube-api-access-tdch9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:23:48.899922 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:23:48.899885 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9c6e38e-b0c3-4446-b432-60f9a3a6685b-extensions-socket-volume\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:23:48.899922 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:23:48.899916 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdch9\" (UniqueName: \"kubernetes.io/projected/f9c6e38e-b0c3-4446-b432-60f9a3a6685b-kube-api-access-tdch9\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:23:48.943419 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:23:48.943385 2566 generic.go:356] "Generic (PLEG): container finished" podID="f9c6e38e-b0c3-4446-b432-60f9a3a6685b" containerID="cac054e4f64cf88dc0b825ad5dee06dc59f239ca742e7ae1eb87dc5e68377916" exitCode=0 Jul 02 19:23:48.943574 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:23:48.943448 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv" Jul 02 19:23:48.943574 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:23:48.943445 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv" event={"ID":"f9c6e38e-b0c3-4446-b432-60f9a3a6685b","Type":"ContainerDied","Data":"cac054e4f64cf88dc0b825ad5dee06dc59f239ca742e7ae1eb87dc5e68377916"} Jul 02 19:23:48.943649 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:23:48.943569 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv" event={"ID":"f9c6e38e-b0c3-4446-b432-60f9a3a6685b","Type":"ContainerDied","Data":"ed61a0b0cc4b1ab6d6f51e95d78abc9f3de72d9198e73624d783e64f9bc6da2a"} Jul 02 19:23:48.943649 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:23:48.943599 2566 scope.go:117] "RemoveContainer" containerID="cac054e4f64cf88dc0b825ad5dee06dc59f239ca742e7ae1eb87dc5e68377916" Jul 02 19:23:48.953527 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:23:48.953507 2566 scope.go:117] "RemoveContainer" containerID="cac054e4f64cf88dc0b825ad5dee06dc59f239ca742e7ae1eb87dc5e68377916" Jul 02 19:23:48.953781 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:23:48.953760 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cac054e4f64cf88dc0b825ad5dee06dc59f239ca742e7ae1eb87dc5e68377916\": container with ID starting with cac054e4f64cf88dc0b825ad5dee06dc59f239ca742e7ae1eb87dc5e68377916 not found: ID does not exist" containerID="cac054e4f64cf88dc0b825ad5dee06dc59f239ca742e7ae1eb87dc5e68377916" Jul 02 19:23:48.953853 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:23:48.953791 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cac054e4f64cf88dc0b825ad5dee06dc59f239ca742e7ae1eb87dc5e68377916"} err="failed to get container status \"cac054e4f64cf88dc0b825ad5dee06dc59f239ca742e7ae1eb87dc5e68377916\": rpc error: code = NotFound desc = could not find container \"cac054e4f64cf88dc0b825ad5dee06dc59f239ca742e7ae1eb87dc5e68377916\": container with ID starting with cac054e4f64cf88dc0b825ad5dee06dc59f239ca742e7ae1eb87dc5e68377916 not found: ID does not exist" Jul 02 19:23:48.961014 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:23:48.960988 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv"] Jul 02 19:23:48.964655 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:23:48.964633 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-54jgv"] Jul 02 19:23:50.844077 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:23:50.844046 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9c6e38e-b0c3-4446-b432-60f9a3a6685b" path="/var/lib/kubelet/pods/f9c6e38e-b0c3-4446-b432-60f9a3a6685b/volumes" Jul 02 19:24:46.707826 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:46.707737 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r62zd"] Jul 02 19:24:46.708249 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:46.708094 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9c6e38e-b0c3-4446-b432-60f9a3a6685b" containerName="manager" Jul 02 19:24:46.708249 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:46.708107 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9c6e38e-b0c3-4446-b432-60f9a3a6685b" containerName="manager" Jul 02 19:24:46.708249 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:46.708120 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d93829de-3bcf-48ee-96af-39473b57055a" containerName="authorino" Jul 02 19:24:46.708249 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:46.708126 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d93829de-3bcf-48ee-96af-39473b57055a" containerName="authorino" Jul 02 19:24:46.708249 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:46.708188 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9c6e38e-b0c3-4446-b432-60f9a3a6685b" containerName="manager" Jul 02 19:24:46.708249 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:46.708195 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d93829de-3bcf-48ee-96af-39473b57055a" containerName="authorino" Jul 02 19:24:46.712260 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:46.712244 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r62zd" Jul 02 19:24:46.714821 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:46.714799 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8jxtl\"" Jul 02 19:24:46.728588 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:46.728560 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r62zd"] Jul 02 19:24:46.783535 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:46.783496 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbqc4\" (UniqueName: \"kubernetes.io/projected/7e58977e-8f80-41e2-b58c-76cc244d58f9-kube-api-access-cbqc4\") pod \"kuadrant-operator-controller-manager-86878c769f-r62zd\" (UID: \"7e58977e-8f80-41e2-b58c-76cc244d58f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r62zd" Jul 02 19:24:46.783726 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:46.783558 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e58977e-8f80-41e2-b58c-76cc244d58f9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-r62zd\" (UID: \"7e58977e-8f80-41e2-b58c-76cc244d58f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r62zd" Jul 02 19:24:46.884185 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:46.884156 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e58977e-8f80-41e2-b58c-76cc244d58f9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-r62zd\" (UID: \"7e58977e-8f80-41e2-b58c-76cc244d58f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r62zd" Jul 02 19:24:46.884357 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:46.884232 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbqc4\" (UniqueName: \"kubernetes.io/projected/7e58977e-8f80-41e2-b58c-76cc244d58f9-kube-api-access-cbqc4\") pod \"kuadrant-operator-controller-manager-86878c769f-r62zd\" (UID: \"7e58977e-8f80-41e2-b58c-76cc244d58f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r62zd" Jul 02 19:24:46.884628 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:46.884609 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e58977e-8f80-41e2-b58c-76cc244d58f9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-r62zd\" (UID: \"7e58977e-8f80-41e2-b58c-76cc244d58f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r62zd" Jul 02 19:24:46.894181 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:46.894157 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbqc4\" (UniqueName: \"kubernetes.io/projected/7e58977e-8f80-41e2-b58c-76cc244d58f9-kube-api-access-cbqc4\") pod \"kuadrant-operator-controller-manager-86878c769f-r62zd\" (UID: \"7e58977e-8f80-41e2-b58c-76cc244d58f9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r62zd" Jul 02 19:24:47.021359 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:47.021322 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r62zd" Jul 02 19:24:47.168640 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:47.168611 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r62zd"] Jul 02 19:24:47.171035 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:24:47.171001 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e58977e_8f80_41e2_b58c_76cc244d58f9.slice/crio-78da65310096c23dd2b0a3317c6c1319a11485ca891db83d9aa2058fd467aa0c WatchSource:0}: Error finding container 78da65310096c23dd2b0a3317c6c1319a11485ca891db83d9aa2058fd467aa0c: Status 404 returned error can't find the container with id 78da65310096c23dd2b0a3317c6c1319a11485ca891db83d9aa2058fd467aa0c Jul 02 19:24:47.218918 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:47.218893 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r62zd" event={"ID":"7e58977e-8f80-41e2-b58c-76cc244d58f9","Type":"ContainerStarted","Data":"78da65310096c23dd2b0a3317c6c1319a11485ca891db83d9aa2058fd467aa0c"} Jul 02 19:24:48.225260 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:48.225223 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r62zd" event={"ID":"7e58977e-8f80-41e2-b58c-76cc244d58f9","Type":"ContainerStarted","Data":"5c242a0b51668846cd78cb0b4277e1da9be40b3388b09a97dab1d0beed1d520a"} Jul 02 19:24:48.225635 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:48.225304 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r62zd" Jul 02 19:24:48.247340 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:48.247292 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r62zd" podStartSLOduration=2.247275446 podStartE2EDuration="2.247275446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:24:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:24:48.244187529 +0000 UTC m=+1168.171196243" watchObservedRunningTime="2026-07-02 19:24:48.247275446 +0000 UTC m=+1168.174284220" Jul 02 19:24:59.232465 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:24:59.232433 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r62zd" Jul 02 19:27:23.975503 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:27:23.975468 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:27:47.256139 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:27:47.256047 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:28:56.110960 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:28:56.110920 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:28:56.211113 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:28:56.211076 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:30:35.203994 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:30:35.203936 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:30:44.079248 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:30:44.079170 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:30:52.349905 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:30:52.349855 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:30:57.473397 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:30:57.473358 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:00.352777 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:00.352741 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:00.448135 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:00.448102 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:00.852149 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:00.852116 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:06.850023 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:06.849985 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:06.926363 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:06.926328 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:07.126578 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:07.126452 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:07.226601 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:07.226569 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:08.360768 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:08.360726 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:08.456723 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:08.456663 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:08.561275 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:08.561236 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:08.885108 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:08.885071 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:09.420327 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:09.420292 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:09.503964 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:09.503916 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:11.314999 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:11.314950 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:13.430105 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:13.430067 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:13.596203 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:13.596162 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:13.701348 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:13.701257 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:13.803941 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:13.803905 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:15.186437 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:15.186402 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:20.003209 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:20.003170 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:20.395205 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:20.395119 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:20.736559 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:20.736525 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:21.420170 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:21.420114 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:21.553232 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:21.553195 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:21.731229 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:21.731198 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:21.937497 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:21.937461 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:22.210541 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:22.210427 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:22.312653 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:22.312603 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:23.680531 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:23.680488 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:26.601520 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:26.601479 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:26.909795 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:26.909697 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:32.956955 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:32.956919 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:33.051791 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:33.051749 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:35.492740 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:35.492703 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:39.148011 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:39.147977 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:39.246829 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:39.246792 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:39.355797 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:39.355762 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:39.449726 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:39.449610 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:40.572004 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:40.571959 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:40.669346 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:40.669305 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:40.820136 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:40.820081 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:40.969090 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:40.969000 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:41.329349 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:41.329305 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:41.429201 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:41.429166 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:43.176553 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:43.176520 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:45.034983 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:45.034948 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:45.226386 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:45.226354 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:45.420907 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:45.420815 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:45.504003 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:45.503968 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:46.877252 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:46.877212 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:51.321752 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:51.321713 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:51.426244 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:51.426209 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:51.808636 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:51.808603 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:51.904808 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:51.904777 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:52.006426 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:52.006394 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:53.403584 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:53.403543 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:57.688603 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:57.688571 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:59.284611 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:59.284573 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:59.392433 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:59.392378 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:59.494056 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:59.493988 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:59.586618 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:59.586485 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:31:59.998580 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:31:59.998542 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:32:26.006894 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:32:26.006858 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:32:32.307736 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:32:32.307696 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:32:32.415596 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:32:32.415554 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:32:32.508470 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:32:32.508432 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:32:48.462740 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:32:48.462663 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:32:54.373145 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:32:54.373096 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:32:54.515468 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:32:54.515436 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:32:54.652758 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:32:54.652654 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:32:58.400634 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:32:58.400590 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:32:58.462926 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:32:58.462897 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:05.178580 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:05.178546 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:06.002290 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:06.002247 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:11.826122 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:11.826082 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:11.939651 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:11.939617 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:12.036918 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:12.036862 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:12.137844 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:12.137745 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:12.907462 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:12.907408 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:13.502760 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:13.502726 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:13.602570 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:13.602512 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:14.745331 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:14.745298 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:20.501987 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:20.501953 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:20.570714 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:20.570663 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:21.172637 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:21.172597 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:27.099110 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:27.099033 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:27.215194 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:27.213376 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:27.430798 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:27.430707 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:27.735048 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:27.735018 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:27.950848 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:27.950808 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:30.754558 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:30.754469 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:30.850003 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:30.849966 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:30.946851 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:30.946817 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:31.333717 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:31.333658 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:31.450992 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:31.450953 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:31.552527 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:31.552491 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:35.231380 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:35.231322 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:35.905978 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:35.905944 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:35.991150 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:35.991114 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:36.313156 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:36.313122 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:42.194564 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:42.194477 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:42.293877 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:42.293840 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:42.594442 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:42.594409 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:42.920489 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:42.920396 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:42.977837 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:42.977804 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:48.933922 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:48.933863 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:49.037446 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:49.037412 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:49.212415 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:49.212311 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:49.433029 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:49.432986 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:49.564888 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:49.564854 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:49.660518 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:49.660488 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:49.769222 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:49.769184 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:55.621326 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:55.621284 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:55.735581 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:55.735541 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:55.848842 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:55.848804 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:33:56.014065 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:33:56.014026 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:34:01.887925 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:01.887891 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:34:01.988217 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:01.988184 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:34:08.393974 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:08.393940 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:34:08.768722 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:08.768661 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:34:08.945297 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:08.945256 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:34:11.980978 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:11.980944 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:34:12.079132 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:12.079078 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:34:12.181468 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:12.181402 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:34:12.478193 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:12.478146 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:34:12.646403 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:12.646369 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:34:12.744410 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:12.744319 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:34:16.519940 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:16.519897 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:34:26.907371 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:26.907328 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:34:27.023138 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:27.023103 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:34:33.431377 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:33.431336 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:34:33.536596 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:33.536561 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:34:39.608263 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.608229 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2"] Jul 02 19:34:39.611851 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.611831 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:39.617787 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.617765 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-65c4b835\"/\"kube-root-ca.crt\"" Jul 02 19:34:39.618047 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.618031 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-65c4b835\"/\"default-dockercfg-vqxj9\"" Jul 02 19:34:39.618489 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.618132 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-65c4b835\"/\"openshift-service-ca.crt\"" Jul 02 19:34:39.618489 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.618209 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-65c4b835\"/\"e2e-lifecycle-model-65c4b835-kserve-self-signed-certs\"" Jul 02 19:34:39.626102 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.626080 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2"] Jul 02 19:34:39.644830 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.644801 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:34:39.749582 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.749548 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-dshm\") pod \"e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:39.749582 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.749587 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e76b788-40c0-443d-aff5-5104ef8d9737-tls-certs\") pod \"e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:39.749802 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.749603 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-home\") pod \"e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:39.749802 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.749747 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-model-cache\") pod \"e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:39.749802 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.749775 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phdjz\" (UniqueName: \"kubernetes.io/projected/6e76b788-40c0-443d-aff5-5104ef8d9737-kube-api-access-phdjz\") pod \"e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:39.749802 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.749795 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-tmp-dir\") pod \"e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:39.850450 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.850410 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-dshm\") pod \"e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:39.850450 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.850450 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e76b788-40c0-443d-aff5-5104ef8d9737-tls-certs\") pod \"e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:39.850649 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.850469 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-home\") pod \"e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:39.850649 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.850541 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-model-cache\") pod \"e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:39.850649 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.850573 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phdjz\" (UniqueName: \"kubernetes.io/projected/6e76b788-40c0-443d-aff5-5104ef8d9737-kube-api-access-phdjz\") pod \"e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:39.850805 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.850728 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-tmp-dir\") pod \"e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:39.850974 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.850949 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-home\") pod \"e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:39.850974 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.850967 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-model-cache\") pod \"e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:39.851112 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.851065 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-tmp-dir\") pod \"e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:39.852719 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.852699 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-dshm\") pod \"e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:39.852998 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.852981 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e76b788-40c0-443d-aff5-5104ef8d9737-tls-certs\") pod \"e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:39.859162 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.859107 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phdjz\" (UniqueName: \"kubernetes.io/projected/6e76b788-40c0-443d-aff5-5104ef8d9737-kube-api-access-phdjz\") pod \"e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:39.921274 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:39.921247 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:40.054183 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:40.054157 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2"] Jul 02 19:34:40.055450 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:34:40.055421 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e76b788_40c0_443d_aff5_5104ef8d9737.slice/crio-5c584d0fc2acb29c9452a215bada025e8d96182c35928cc3051e4276ad27b2b9 WatchSource:0}: Error finding container 5c584d0fc2acb29c9452a215bada025e8d96182c35928cc3051e4276ad27b2b9: Status 404 returned error can't find the container with id 5c584d0fc2acb29c9452a215bada025e8d96182c35928cc3051e4276ad27b2b9 Jul 02 19:34:40.179969 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:40.179933 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" event={"ID":"6e76b788-40c0-443d-aff5-5104ef8d9737","Type":"ContainerStarted","Data":"20c6a54a61437531f4699e3268204326049474bfe10f677c7ba4cec960dade7f"} Jul 02 19:34:40.180145 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:40.179982 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" event={"ID":"6e76b788-40c0-443d-aff5-5104ef8d9737","Type":"ContainerStarted","Data":"5c584d0fc2acb29c9452a215bada025e8d96182c35928cc3051e4276ad27b2b9"} Jul 02 19:34:40.204893 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:40.204837 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" podStartSLOduration=1.204820661 podStartE2EDuration="1.204820661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:34:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:34:40.203480278 +0000 UTC m=+1760.130488983" watchObservedRunningTime="2026-07-02 19:34:40.204820661 +0000 UTC m=+1760.131829367" Jul 02 19:34:49.922188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:49.922153 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:49.922188 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:49.922206 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:49.934510 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:49.934483 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:34:50.240092 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:34:50.240059 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:35:04.423140 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:04.423099 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:35:04.869521 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:04.869485 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:35:10.578952 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:10.578903 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:35:15.539139 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:15.539055 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2"] Jul 02 19:35:15.635891 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:15.635849 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:35:30.561021 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:30.560960 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" podUID="6e76b788-40c0-443d-aff5-5104ef8d9737" containerName="main" containerID="cri-o://20c6a54a61437531f4699e3268204326049474bfe10f677c7ba4cec960dade7f" gracePeriod=45 Jul 02 19:35:30.806281 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:30.806257 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:35:30.906974 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:30.906888 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e76b788-40c0-443d-aff5-5104ef8d9737-tls-certs\") pod \"6e76b788-40c0-443d-aff5-5104ef8d9737\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " Jul 02 19:35:30.906974 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:30.906932 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-model-cache\") pod \"6e76b788-40c0-443d-aff5-5104ef8d9737\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " Jul 02 19:35:30.906974 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:30.906972 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-home\") pod \"6e76b788-40c0-443d-aff5-5104ef8d9737\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " Jul 02 19:35:30.907261 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:30.907018 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phdjz\" (UniqueName: \"kubernetes.io/projected/6e76b788-40c0-443d-aff5-5104ef8d9737-kube-api-access-phdjz\") pod \"6e76b788-40c0-443d-aff5-5104ef8d9737\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " Jul 02 19:35:30.907261 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:30.907040 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-tmp-dir\") pod \"6e76b788-40c0-443d-aff5-5104ef8d9737\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " Jul 02 19:35:30.907261 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:30.907058 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-dshm\") pod \"6e76b788-40c0-443d-aff5-5104ef8d9737\" (UID: \"6e76b788-40c0-443d-aff5-5104ef8d9737\") " Jul 02 19:35:30.907388 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:30.907278 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-home" (OuterVolumeSpecName: "home") pod "6e76b788-40c0-443d-aff5-5104ef8d9737" (UID: "6e76b788-40c0-443d-aff5-5104ef8d9737"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:35:30.907388 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:30.907291 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-model-cache" (OuterVolumeSpecName: "model-cache") pod "6e76b788-40c0-443d-aff5-5104ef8d9737" (UID: "6e76b788-40c0-443d-aff5-5104ef8d9737"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:35:30.907388 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:30.907302 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6e76b788-40c0-443d-aff5-5104ef8d9737" (UID: "6e76b788-40c0-443d-aff5-5104ef8d9737"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:35:30.909184 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:30.909155 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-dshm" (OuterVolumeSpecName: "dshm") pod "6e76b788-40c0-443d-aff5-5104ef8d9737" (UID: "6e76b788-40c0-443d-aff5-5104ef8d9737"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:35:30.909281 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:30.909230 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e76b788-40c0-443d-aff5-5104ef8d9737-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6e76b788-40c0-443d-aff5-5104ef8d9737" (UID: "6e76b788-40c0-443d-aff5-5104ef8d9737"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:35:30.909281 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:30.909232 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e76b788-40c0-443d-aff5-5104ef8d9737-kube-api-access-phdjz" (OuterVolumeSpecName: "kube-api-access-phdjz") pod "6e76b788-40c0-443d-aff5-5104ef8d9737" (UID: "6e76b788-40c0-443d-aff5-5104ef8d9737"). InnerVolumeSpecName "kube-api-access-phdjz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:35:31.008429 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:31.008386 2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e76b788-40c0-443d-aff5-5104ef8d9737-tls-certs\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:35:31.008429 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:31.008420 2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-model-cache\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:35:31.008429 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:31.008433 2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-home\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:35:31.008703 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:31.008446 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phdjz\" (UniqueName: \"kubernetes.io/projected/6e76b788-40c0-443d-aff5-5104ef8d9737-kube-api-access-phdjz\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:35:31.008703 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:31.008459 2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-tmp-dir\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:35:31.008703 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:31.008472 2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e76b788-40c0-443d-aff5-5104ef8d9737-dshm\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:35:31.436912 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:31.436876 2566 generic.go:356] "Generic (PLEG): container finished" podID="6e76b788-40c0-443d-aff5-5104ef8d9737" containerID="20c6a54a61437531f4699e3268204326049474bfe10f677c7ba4cec960dade7f" exitCode=0 Jul 02 19:35:31.437092 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:31.436934 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" Jul 02 19:35:31.437092 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:31.436962 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" event={"ID":"6e76b788-40c0-443d-aff5-5104ef8d9737","Type":"ContainerDied","Data":"20c6a54a61437531f4699e3268204326049474bfe10f677c7ba4cec960dade7f"} Jul 02 19:35:31.437092 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:31.437006 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2" event={"ID":"6e76b788-40c0-443d-aff5-5104ef8d9737","Type":"ContainerDied","Data":"5c584d0fc2acb29c9452a215bada025e8d96182c35928cc3051e4276ad27b2b9"} Jul 02 19:35:31.437092 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:31.437024 2566 scope.go:117] "RemoveContainer" containerID="20c6a54a61437531f4699e3268204326049474bfe10f677c7ba4cec960dade7f" Jul 02 19:35:31.446875 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:31.446799 2566 scope.go:117] "RemoveContainer" containerID="20c6a54a61437531f4699e3268204326049474bfe10f677c7ba4cec960dade7f" Jul 02 19:35:31.448653 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:35:31.448630 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20c6a54a61437531f4699e3268204326049474bfe10f677c7ba4cec960dade7f\": container with ID starting with 20c6a54a61437531f4699e3268204326049474bfe10f677c7ba4cec960dade7f not found: ID does not exist" containerID="20c6a54a61437531f4699e3268204326049474bfe10f677c7ba4cec960dade7f" Jul 02 19:35:31.448776 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:31.448660 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20c6a54a61437531f4699e3268204326049474bfe10f677c7ba4cec960dade7f"} err="failed to get container status \"20c6a54a61437531f4699e3268204326049474bfe10f677c7ba4cec960dade7f\": rpc error: code = NotFound desc = could not find container \"20c6a54a61437531f4699e3268204326049474bfe10f677c7ba4cec960dade7f\": container with ID starting with 20c6a54a61437531f4699e3268204326049474bfe10f677c7ba4cec960dade7f not found: ID does not exist" Jul 02 19:35:31.458406 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:31.458382 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2"] Jul 02 19:35:31.463789 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:31.463765 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-65c4b835/e2e-lifecycle-model-65c4b835-kserve-b585f75c9-47cm2"] Jul 02 19:35:32.844401 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:32.844368 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e76b788-40c0-443d-aff5-5104ef8d9737" path="/var/lib/kubelet/pods/6e76b788-40c0-443d-aff5-5104ef8d9737/volumes" Jul 02 19:35:52.142263 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:52.142220 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:35:52.613490 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:52.613452 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:35:58.176496 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:35:58.176459 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:03.871536 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:03.871496 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:03.938698 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:03.938641 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:04.175342 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:04.175257 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:04.432332 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:04.432253 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:05.624546 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:05.624506 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:05.722411 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:05.722379 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:05.823755 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:05.823719 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:06.591766 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:06.591725 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:06.690663 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:06.690623 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:06.793340 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:06.793305 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:08.387378 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:08.387337 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:14.351861 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:14.351826 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:14.452050 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:14.452017 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:15.059154 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:15.059120 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:21.042729 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:21.042667 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:21.244048 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:21.244017 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:21.371147 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:21.371114 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:31.815009 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:31.814969 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:32.171043 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:32.170955 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:53.134840 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:53.134800 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:58.855592 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:58.855559 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:58.916195 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:58.916161 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:59.230353 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:59.230316 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:59.571664 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:59.571559 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:36:59.633938 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:36:59.633904 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:02.841511 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:02.841474 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:02.932004 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:02.931971 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:02.994435 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:02.994404 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:03.388515 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:03.388483 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:03.485640 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:03.485603 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:07.369773 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:07.369736 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:07.702148 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:07.702047 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:08.020923 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:08.020882 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:08.396821 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:08.396726 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:10.955502 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:10.955413 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:11.087639 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:11.087599 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:11.188437 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:11.188394 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:11.392745 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:11.392707 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:11.805044 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:11.805010 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:11.904402 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:11.904370 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:15.470434 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:15.470381 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:23.490546 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.490510 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn"] Jul 02 19:37:23.490978 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.490861 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e76b788-40c0-443d-aff5-5104ef8d9737" containerName="main" Jul 02 19:37:23.490978 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.490873 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e76b788-40c0-443d-aff5-5104ef8d9737" containerName="main" Jul 02 19:37:23.490978 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.490958 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e76b788-40c0-443d-aff5-5104ef8d9737" containerName="main" Jul 02 19:37:23.493809 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.493791 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:23.497442 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.497420 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-ef1ca5\"/\"test-model-ef1ca5-kserve-self-signed-certs\"" Jul 02 19:37:23.497570 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.497423 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-ef1ca5\"/\"openshift-service-ca.crt\"" Jul 02 19:37:23.497570 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.497478 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-ef1ca5\"/\"kube-root-ca.crt\"" Jul 02 19:37:23.497570 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.497513 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-ef1ca5\"/\"default-dockercfg-tbhhs\"" Jul 02 19:37:23.506412 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.506390 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn"] Jul 02 19:37:23.506718 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.506696 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fe06eef-5b41-4772-80be-6e9b17ee549b-tls-certs\") pod \"test-model-ef1ca5-kserve-6c67c7675-mw2tn\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:23.506777 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.506762 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p97j\" (UniqueName: \"kubernetes.io/projected/9fe06eef-5b41-4772-80be-6e9b17ee549b-kube-api-access-2p97j\") pod \"test-model-ef1ca5-kserve-6c67c7675-mw2tn\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:23.506818 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.506790 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-home\") pod \"test-model-ef1ca5-kserve-6c67c7675-mw2tn\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:23.506818 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.506807 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-model-cache\") pod \"test-model-ef1ca5-kserve-6c67c7675-mw2tn\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:23.506927 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.506821 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-tmp-dir\") pod \"test-model-ef1ca5-kserve-6c67c7675-mw2tn\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:23.506927 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.506871 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-dshm\") pod \"test-model-ef1ca5-kserve-6c67c7675-mw2tn\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:23.585529 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.585494 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:23.607957 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.607918 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p97j\" (UniqueName: \"kubernetes.io/projected/9fe06eef-5b41-4772-80be-6e9b17ee549b-kube-api-access-2p97j\") pod \"test-model-ef1ca5-kserve-6c67c7675-mw2tn\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:23.607957 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.607960 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-home\") pod \"test-model-ef1ca5-kserve-6c67c7675-mw2tn\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:23.608208 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.607978 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-model-cache\") pod \"test-model-ef1ca5-kserve-6c67c7675-mw2tn\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:23.608208 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.608001 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-tmp-dir\") pod \"test-model-ef1ca5-kserve-6c67c7675-mw2tn\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:23.608208 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.608033 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-dshm\") pod \"test-model-ef1ca5-kserve-6c67c7675-mw2tn\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:23.608208 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.608100 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fe06eef-5b41-4772-80be-6e9b17ee549b-tls-certs\") pod \"test-model-ef1ca5-kserve-6c67c7675-mw2tn\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:23.608428 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.608391 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-home\") pod \"test-model-ef1ca5-kserve-6c67c7675-mw2tn\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:23.608491 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.608459 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-model-cache\") pod \"test-model-ef1ca5-kserve-6c67c7675-mw2tn\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:23.608548 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.608506 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-tmp-dir\") pod \"test-model-ef1ca5-kserve-6c67c7675-mw2tn\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:23.610443 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.610419 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-dshm\") pod \"test-model-ef1ca5-kserve-6c67c7675-mw2tn\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:23.610577 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.610537 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fe06eef-5b41-4772-80be-6e9b17ee549b-tls-certs\") pod \"test-model-ef1ca5-kserve-6c67c7675-mw2tn\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:23.616401 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.616382 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p97j\" (UniqueName: \"kubernetes.io/projected/9fe06eef-5b41-4772-80be-6e9b17ee549b-kube-api-access-2p97j\") pod \"test-model-ef1ca5-kserve-6c67c7675-mw2tn\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:23.803187 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.803081 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:23.936064 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.936015 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn"] Jul 02 19:37:23.938366 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:37:23.938339 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9fe06eef_5b41_4772_80be_6e9b17ee549b.slice/crio-6400eac44ba0da71a82604c3fcde6506418621257427fba05521061d60aad1d6 WatchSource:0}: Error finding container 6400eac44ba0da71a82604c3fcde6506418621257427fba05521061d60aad1d6: Status 404 returned error can't find the container with id 6400eac44ba0da71a82604c3fcde6506418621257427fba05521061d60aad1d6 Jul 02 19:37:23.970918 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:23.970892 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" event={"ID":"9fe06eef-5b41-4772-80be-6e9b17ee549b","Type":"ContainerStarted","Data":"6400eac44ba0da71a82604c3fcde6506418621257427fba05521061d60aad1d6"} Jul 02 19:37:24.309050 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:24.309010 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:24.405504 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:24.405464 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:24.977330 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:24.977293 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" event={"ID":"9fe06eef-5b41-4772-80be-6e9b17ee549b","Type":"ContainerStarted","Data":"0c9336bbabb33fe11f4e72c28b88d2e9c982e94bdc00f6c20dcbfc3d011dc813"} Jul 02 19:37:24.996600 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:24.996540 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" podStartSLOduration=1.996524575 podStartE2EDuration="1.996524575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:37:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:37:24.994047733 +0000 UTC m=+1924.921056446" watchObservedRunningTime="2026-07-02 19:37:24.996524575 +0000 UTC m=+1924.923533289" Jul 02 19:37:32.504263 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:32.504203 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:33.804169 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:33.804129 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:33.804573 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:33.804191 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:33.817023 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:33.816996 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:34.034075 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:34.034033 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:34.040991 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:34.040970 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:37:40.296065 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:40.296033 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:40.397790 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:40.397754 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:37:48.515292 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:37:48.515256 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:08.864580 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:08.864548 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn"] Jul 02 19:38:08.885164 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:08.885123 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:09.779284 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:09.779246 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:09.964829 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:09.964790 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:10.343529 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:10.343490 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:10.452548 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:10.452509 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:10.603882 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:10.603783 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:10.729890 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:10.729855 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:15.422102 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:15.421993 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:23.884574 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:23.884530 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" podUID="9fe06eef-5b41-4772-80be-6e9b17ee549b" containerName="main" containerID="cri-o://0c9336bbabb33fe11f4e72c28b88d2e9c982e94bdc00f6c20dcbfc3d011dc813" gracePeriod=45 Jul 02 19:38:24.029879 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.029832 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" podUID="9fe06eef-5b41-4772-80be-6e9b17ee549b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": dial tcp 10.134.0.45:8000: connect: connection refused" Jul 02 19:38:24.130970 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.130943 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:38:24.273863 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.273829 2566 generic.go:356] "Generic (PLEG): container finished" podID="9fe06eef-5b41-4772-80be-6e9b17ee549b" containerID="0c9336bbabb33fe11f4e72c28b88d2e9c982e94bdc00f6c20dcbfc3d011dc813" exitCode=0 Jul 02 19:38:24.274074 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.273869 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" Jul 02 19:38:24.274074 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.273919 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" event={"ID":"9fe06eef-5b41-4772-80be-6e9b17ee549b","Type":"ContainerDied","Data":"0c9336bbabb33fe11f4e72c28b88d2e9c982e94bdc00f6c20dcbfc3d011dc813"} Jul 02 19:38:24.274074 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.273970 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn" event={"ID":"9fe06eef-5b41-4772-80be-6e9b17ee549b","Type":"ContainerDied","Data":"6400eac44ba0da71a82604c3fcde6506418621257427fba05521061d60aad1d6"} Jul 02 19:38:24.274074 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.273993 2566 scope.go:117] "RemoveContainer" containerID="0c9336bbabb33fe11f4e72c28b88d2e9c982e94bdc00f6c20dcbfc3d011dc813" Jul 02 19:38:24.274764 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.274747 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fe06eef-5b41-4772-80be-6e9b17ee549b-tls-certs\") pod \"9fe06eef-5b41-4772-80be-6e9b17ee549b\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " Jul 02 19:38:24.274818 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.274798 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-home\") pod \"9fe06eef-5b41-4772-80be-6e9b17ee549b\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " Jul 02 19:38:24.274868 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.274858 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-model-cache\") pod \"9fe06eef-5b41-4772-80be-6e9b17ee549b\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " Jul 02 19:38:24.274923 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.274877 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2p97j\" (UniqueName: \"kubernetes.io/projected/9fe06eef-5b41-4772-80be-6e9b17ee549b-kube-api-access-2p97j\") pod \"9fe06eef-5b41-4772-80be-6e9b17ee549b\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " Jul 02 19:38:24.274923 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.274895 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-dshm\") pod \"9fe06eef-5b41-4772-80be-6e9b17ee549b\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " Jul 02 19:38:24.274923 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.274919 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-tmp-dir\") pod \"9fe06eef-5b41-4772-80be-6e9b17ee549b\" (UID: \"9fe06eef-5b41-4772-80be-6e9b17ee549b\") " Jul 02 19:38:24.275143 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.275110 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-model-cache" (OuterVolumeSpecName: "model-cache") pod "9fe06eef-5b41-4772-80be-6e9b17ee549b" (UID: "9fe06eef-5b41-4772-80be-6e9b17ee549b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:38:24.275143 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.275127 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-home" (OuterVolumeSpecName: "home") pod "9fe06eef-5b41-4772-80be-6e9b17ee549b" (UID: "9fe06eef-5b41-4772-80be-6e9b17ee549b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:38:24.275313 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.275261 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9fe06eef-5b41-4772-80be-6e9b17ee549b" (UID: "9fe06eef-5b41-4772-80be-6e9b17ee549b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:38:24.275313 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.275288 2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-model-cache\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:38:24.275313 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.275305 2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-home\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:38:24.277266 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.277240 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-dshm" (OuterVolumeSpecName: "dshm") pod "9fe06eef-5b41-4772-80be-6e9b17ee549b" (UID: "9fe06eef-5b41-4772-80be-6e9b17ee549b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:38:24.277399 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.277372 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fe06eef-5b41-4772-80be-6e9b17ee549b-kube-api-access-2p97j" (OuterVolumeSpecName: "kube-api-access-2p97j") pod "9fe06eef-5b41-4772-80be-6e9b17ee549b" (UID: "9fe06eef-5b41-4772-80be-6e9b17ee549b"). InnerVolumeSpecName "kube-api-access-2p97j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:38:24.277462 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.277403 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fe06eef-5b41-4772-80be-6e9b17ee549b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9fe06eef-5b41-4772-80be-6e9b17ee549b" (UID: "9fe06eef-5b41-4772-80be-6e9b17ee549b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:38:24.294076 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.294051 2566 scope.go:117] "RemoveContainer" containerID="0c9336bbabb33fe11f4e72c28b88d2e9c982e94bdc00f6c20dcbfc3d011dc813" Jul 02 19:38:24.294350 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:38:24.294333 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c9336bbabb33fe11f4e72c28b88d2e9c982e94bdc00f6c20dcbfc3d011dc813\": container with ID starting with 0c9336bbabb33fe11f4e72c28b88d2e9c982e94bdc00f6c20dcbfc3d011dc813 not found: ID does not exist" containerID="0c9336bbabb33fe11f4e72c28b88d2e9c982e94bdc00f6c20dcbfc3d011dc813" Jul 02 19:38:24.294401 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.294360 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c9336bbabb33fe11f4e72c28b88d2e9c982e94bdc00f6c20dcbfc3d011dc813"} err="failed to get container status \"0c9336bbabb33fe11f4e72c28b88d2e9c982e94bdc00f6c20dcbfc3d011dc813\": rpc error: code = NotFound desc = could not find container \"0c9336bbabb33fe11f4e72c28b88d2e9c982e94bdc00f6c20dcbfc3d011dc813\": container with ID starting with 0c9336bbabb33fe11f4e72c28b88d2e9c982e94bdc00f6c20dcbfc3d011dc813 not found: ID does not exist" Jul 02 19:38:24.376794 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.376760 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2p97j\" (UniqueName: \"kubernetes.io/projected/9fe06eef-5b41-4772-80be-6e9b17ee549b-kube-api-access-2p97j\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:38:24.376794 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.376789 2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-dshm\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:38:24.376794 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.376802 2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fe06eef-5b41-4772-80be-6e9b17ee549b-tmp-dir\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:38:24.377026 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.376811 2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fe06eef-5b41-4772-80be-6e9b17ee549b-tls-certs\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:38:24.597004 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.596972 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn"] Jul 02 19:38:24.601377 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.601355 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-ef1ca5/test-model-ef1ca5-kserve-6c67c7675-mw2tn"] Jul 02 19:38:24.843570 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:24.843493 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fe06eef-5b41-4772-80be-6e9b17ee549b" path="/var/lib/kubelet/pods/9fe06eef-5b41-4772-80be-6e9b17ee549b/volumes" Jul 02 19:38:25.657720 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:25.657670 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:25.708971 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:25.708935 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:52.251454 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:52.251414 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:52.511420 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:52.511332 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:52.750614 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:52.750575 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:53.087214 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:53.087165 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:53.497211 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:53.497175 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:53.584712 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:53.584660 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:53.685530 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:53.685494 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:53.818035 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:53.817945 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:59.125435 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:59.125398 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:59.225868 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:59.225819 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:59.379309 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:59.379225 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:59.601839 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:59.601792 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:38:59.941449 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:38:59.941419 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:00.276650 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:00.276616 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:00.538082 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:00.537976 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:03.475133 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:03.475097 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:03.554548 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:03.554513 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:03.708116 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:03.708072 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:03.936348 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:03.936254 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:04.186220 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:04.186184 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:04.286782 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:04.286746 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:07.986991 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:07.986961 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:08.448334 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:08.448236 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:08.765796 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:08.765768 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:09.337820 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:09.337785 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:11.531606 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:11.531570 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:11.634548 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:11.634497 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:11.765525 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:11.765485 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:12.293729 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:12.293692 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:12.395621 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:12.395587 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:12.827153 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:12.827111 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:16.048860 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:16.048817 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:24.163544 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:24.163508 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:24.630922 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:24.630882 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:24.731053 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:24.731015 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:24.896832 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:24.896748 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:27.782107 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:27.782061 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:27.883168 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:27.883131 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:30.273807 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:30.273771 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:30.663256 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:30.663167 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:30.763309 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:30.763272 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:30.821458 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:30.821418 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:30.956071 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:30.955979 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:31.344829 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:31.344787 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:31.436693 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:31.436633 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:31.536824 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:31.536787 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:31.637047 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:31.636952 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:37.217384 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:37.217329 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:37.317686 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:37.317651 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:37.481406 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:37.481375 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:37.731229 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:37.731196 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:38.064719 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:38.064650 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:38.376545 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:38.376460 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:38.592863 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:38.592821 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:41.700537 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:41.700502 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:41.779939 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:41.779906 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:41.857306 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:41.857270 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:42.094618 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:42.094537 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:42.357445 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:42.357353 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:42.455040 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:42.455006 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:46.228585 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:46.228542 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:46.539764 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:46.539657 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:46.843558 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:46.843458 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:46.973056 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:46.973014 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:49.764699 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:49.764630 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:49.842830 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:49.842792 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:49.905083 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:49.905045 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:50.236372 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:50.236336 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:50.337210 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:50.337168 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:50.445768 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:50.445724 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:39:54.268870 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:39:54.268833 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:40:03.107034 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:03.106986 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:40:03.208457 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:03.208424 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:40:11.890149 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:11.890112 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:40:11.987239 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:11.987200 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:40:12.288231 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.288189 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg"] Jul 02 19:40:12.288534 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.288520 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fe06eef-5b41-4772-80be-6e9b17ee549b" containerName="main" Jul 02 19:40:12.288534 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.288534 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fe06eef-5b41-4772-80be-6e9b17ee549b" containerName="main" Jul 02 19:40:12.288651 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.288599 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fe06eef-5b41-4772-80be-6e9b17ee549b" containerName="main" Jul 02 19:40:12.291532 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.291511 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:12.294052 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.294020 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4a101a\"/\"default-dockercfg-nbl6m\"" Jul 02 19:40:12.294176 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.294021 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4a101a\"/\"openshift-service-ca.crt\"" Jul 02 19:40:12.294746 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.294729 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4a101a\"/\"kube-root-ca.crt\"" Jul 02 19:40:12.294820 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.294805 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4a101a\"/\"auth-test-model-4a101a-kserve-self-signed-certs\"" Jul 02 19:40:12.300149 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.300128 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg"] Jul 02 19:40:12.316501 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.316479 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:40:12.441034 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.440989 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gl8v\" (UniqueName: \"kubernetes.io/projected/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-kube-api-access-8gl8v\") pod \"auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:12.441034 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.441031 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-dshm\") pod \"auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:12.441325 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.441101 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-tmp-dir\") pod \"auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:12.441325 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.441147 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-home\") pod \"auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:12.441325 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.441172 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-model-cache\") pod \"auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:12.441325 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.441190 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-tls-certs\") pod \"auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:12.542659 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.542570 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-tmp-dir\") pod \"auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:12.542659 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.542613 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-home\") pod \"auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:12.542659 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.542633 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-model-cache\") pod \"auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:12.542659 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.542653 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-tls-certs\") pod \"auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:12.543028 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.542750 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gl8v\" (UniqueName: \"kubernetes.io/projected/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-kube-api-access-8gl8v\") pod \"auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:12.543028 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.542781 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-dshm\") pod \"auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:12.543162 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.543037 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-tmp-dir\") pod \"auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:12.543162 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.543068 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-model-cache\") pod \"auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:12.543162 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.543128 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-home\") pod \"auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:12.544974 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.544948 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-dshm\") pod \"auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:12.545256 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.545235 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-tls-certs\") pod \"auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:12.550870 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.550849 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gl8v\" (UniqueName: \"kubernetes.io/projected/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-kube-api-access-8gl8v\") pod \"auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:12.601459 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.601420 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:12.735489 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.735406 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg"] Jul 02 19:40:12.738657 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:40:12.738629 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a8f93e6_97a4_4b67_87c6_801d4f7900c8.slice/crio-e09429496f4ebd1c8e286a843d21f6a8a4ead2ff89fbe48ee201f5aecb206325 WatchSource:0}: Error finding container e09429496f4ebd1c8e286a843d21f6a8a4ead2ff89fbe48ee201f5aecb206325: Status 404 returned error can't find the container with id e09429496f4ebd1c8e286a843d21f6a8a4ead2ff89fbe48ee201f5aecb206325 Jul 02 19:40:12.797279 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:12.797253 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" event={"ID":"7a8f93e6-97a4-4b67-87c6-801d4f7900c8","Type":"ContainerStarted","Data":"e09429496f4ebd1c8e286a843d21f6a8a4ead2ff89fbe48ee201f5aecb206325"} Jul 02 19:40:13.803515 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:13.803476 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" event={"ID":"7a8f93e6-97a4-4b67-87c6-801d4f7900c8","Type":"ContainerStarted","Data":"c47e57a065a4df4d15ad06140aa3af6e88dad74eb6087c43c6632f4333534523"} Jul 02 19:40:13.819786 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:13.819730 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" podStartSLOduration=1.819716321 podStartE2EDuration="1.819716321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:40:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:40:13.818918069 +0000 UTC m=+2093.745926784" watchObservedRunningTime="2026-07-02 19:40:13.819716321 +0000 UTC m=+2093.746725033" Jul 02 19:40:22.602292 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:22.602242 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:22.602292 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:22.602304 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:22.618560 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:22.618534 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:40:22.861174 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:40:22.861086 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:41:04.884152 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:04.884114 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz"] Jul 02 19:41:04.887890 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:04.887868 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:04.891506 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:04.891477 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f80343\"/\"auth-test-model-f80343-kserve-self-signed-certs\"" Jul 02 19:41:04.891640 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:04.891477 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f80343\"/\"kube-root-ca.crt\"" Jul 02 19:41:04.891640 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:04.891510 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f80343\"/\"default-dockercfg-ktxdg\"" Jul 02 19:41:04.891640 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:04.891518 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f80343\"/\"openshift-service-ca.crt\"" Jul 02 19:41:04.895368 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:04.895346 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz"] Jul 02 19:41:04.918627 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:04.918590 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:41:04.929711 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:04.929658 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-home\") pod \"auth-test-model-f80343-kserve-6ff7544875-z2jpz\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:04.929843 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:04.929715 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df2fb747-91cc-4c36-a331-59f60e20a2e2-tls-certs\") pod \"auth-test-model-f80343-kserve-6ff7544875-z2jpz\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:04.929843 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:04.929791 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-tmp-dir\") pod \"auth-test-model-f80343-kserve-6ff7544875-z2jpz\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:04.929843 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:04.929817 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc98b\" (UniqueName: \"kubernetes.io/projected/df2fb747-91cc-4c36-a331-59f60e20a2e2-kube-api-access-vc98b\") pod \"auth-test-model-f80343-kserve-6ff7544875-z2jpz\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:04.929970 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:04.929853 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-dshm\") pod \"auth-test-model-f80343-kserve-6ff7544875-z2jpz\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:04.929970 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:04.929879 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-model-cache\") pod \"auth-test-model-f80343-kserve-6ff7544875-z2jpz\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:05.030964 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:05.030926 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-home\") pod \"auth-test-model-f80343-kserve-6ff7544875-z2jpz\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:05.030964 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:05.030962 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df2fb747-91cc-4c36-a331-59f60e20a2e2-tls-certs\") pod \"auth-test-model-f80343-kserve-6ff7544875-z2jpz\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:05.031235 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:05.030984 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-tmp-dir\") pod \"auth-test-model-f80343-kserve-6ff7544875-z2jpz\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:05.031235 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:05.031003 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc98b\" (UniqueName: \"kubernetes.io/projected/df2fb747-91cc-4c36-a331-59f60e20a2e2-kube-api-access-vc98b\") pod \"auth-test-model-f80343-kserve-6ff7544875-z2jpz\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:05.031235 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:05.031029 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-dshm\") pod \"auth-test-model-f80343-kserve-6ff7544875-z2jpz\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:05.031235 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:05.031051 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-model-cache\") pod \"auth-test-model-f80343-kserve-6ff7544875-z2jpz\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:05.031459 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:05.031420 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-home\") pod \"auth-test-model-f80343-kserve-6ff7544875-z2jpz\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:05.031459 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:05.031447 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-tmp-dir\") pod \"auth-test-model-f80343-kserve-6ff7544875-z2jpz\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:05.031531 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:05.031498 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-model-cache\") pod \"auth-test-model-f80343-kserve-6ff7544875-z2jpz\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:05.033258 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:05.033239 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-dshm\") pod \"auth-test-model-f80343-kserve-6ff7544875-z2jpz\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:05.033524 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:05.033507 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df2fb747-91cc-4c36-a331-59f60e20a2e2-tls-certs\") pod \"auth-test-model-f80343-kserve-6ff7544875-z2jpz\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:05.039656 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:05.039632 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc98b\" (UniqueName: \"kubernetes.io/projected/df2fb747-91cc-4c36-a331-59f60e20a2e2-kube-api-access-vc98b\") pod \"auth-test-model-f80343-kserve-6ff7544875-z2jpz\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:05.198875 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:05.198782 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:05.335936 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:05.335909 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz"] Jul 02 19:41:05.337653 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:41:05.337626 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf2fb747_91cc_4c36_a331_59f60e20a2e2.slice/crio-f5cc71bfc80e3c4ebebe20f088a6a9e90cbc5bb9147e861683cdc7d07624ac9c WatchSource:0}: Error finding container f5cc71bfc80e3c4ebebe20f088a6a9e90cbc5bb9147e861683cdc7d07624ac9c: Status 404 returned error can't find the container with id f5cc71bfc80e3c4ebebe20f088a6a9e90cbc5bb9147e861683cdc7d07624ac9c Jul 02 19:41:06.060955 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:06.060923 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" event={"ID":"df2fb747-91cc-4c36-a331-59f60e20a2e2","Type":"ContainerStarted","Data":"e3002a5d8ff39a15cae10f1aed9d4b025a8829512775dbd9a0a9a2a819e175a6"} Jul 02 19:41:06.061320 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:06.060966 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" event={"ID":"df2fb747-91cc-4c36-a331-59f60e20a2e2","Type":"ContainerStarted","Data":"f5cc71bfc80e3c4ebebe20f088a6a9e90cbc5bb9147e861683cdc7d07624ac9c"} Jul 02 19:41:06.077807 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:06.077760 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" podStartSLOduration=2.077744635 podStartE2EDuration="2.077744635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:41:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:41:06.07606315 +0000 UTC m=+2146.003072110" watchObservedRunningTime="2026-07-02 19:41:06.077744635 +0000 UTC m=+2146.004753388" Jul 02 19:41:15.199647 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:15.199553 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:15.199647 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:15.199611 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:15.212318 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:15.212291 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:41:16.119746 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:41:16.119718 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:42:44.467756 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:42:44.467637 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:42:45.774226 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:42:45.774173 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:42:54.258524 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:42:54.258482 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg"] Jul 02 19:42:54.272840 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:42:54.272807 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:42:55.164575 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:42:55.164542 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz"] Jul 02 19:42:55.187083 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:42:55.187053 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:43:03.809831 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:03.809788 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:43:09.278619 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.278579 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" podUID="7a8f93e6-97a4-4b67-87c6-801d4f7900c8" containerName="main" containerID="cri-o://c47e57a065a4df4d15ad06140aa3af6e88dad74eb6087c43c6632f4333534523" gracePeriod=45 Jul 02 19:43:09.526523 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.526493 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:43:09.662866 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.662783 2566 generic.go:356] "Generic (PLEG): container finished" podID="7a8f93e6-97a4-4b67-87c6-801d4f7900c8" containerID="c47e57a065a4df4d15ad06140aa3af6e88dad74eb6087c43c6632f4333534523" exitCode=0 Jul 02 19:43:09.662866 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.662833 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" Jul 02 19:43:09.662866 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.662839 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" event={"ID":"7a8f93e6-97a4-4b67-87c6-801d4f7900c8","Type":"ContainerDied","Data":"c47e57a065a4df4d15ad06140aa3af6e88dad74eb6087c43c6632f4333534523"} Jul 02 19:43:09.663101 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.662878 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg" event={"ID":"7a8f93e6-97a4-4b67-87c6-801d4f7900c8","Type":"ContainerDied","Data":"e09429496f4ebd1c8e286a843d21f6a8a4ead2ff89fbe48ee201f5aecb206325"} Jul 02 19:43:09.663101 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.662902 2566 scope.go:117] "RemoveContainer" containerID="c47e57a065a4df4d15ad06140aa3af6e88dad74eb6087c43c6632f4333534523" Jul 02 19:43:09.671388 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.671368 2566 scope.go:117] "RemoveContainer" containerID="c47e57a065a4df4d15ad06140aa3af6e88dad74eb6087c43c6632f4333534523" Jul 02 19:43:09.671732 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:43:09.671651 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c47e57a065a4df4d15ad06140aa3af6e88dad74eb6087c43c6632f4333534523\": container with ID starting with c47e57a065a4df4d15ad06140aa3af6e88dad74eb6087c43c6632f4333534523 not found: ID does not exist" containerID="c47e57a065a4df4d15ad06140aa3af6e88dad74eb6087c43c6632f4333534523" Jul 02 19:43:09.671732 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.671698 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c47e57a065a4df4d15ad06140aa3af6e88dad74eb6087c43c6632f4333534523"} err="failed to get container status \"c47e57a065a4df4d15ad06140aa3af6e88dad74eb6087c43c6632f4333534523\": rpc error: code = NotFound desc = could not find container \"c47e57a065a4df4d15ad06140aa3af6e88dad74eb6087c43c6632f4333534523\": container with ID starting with c47e57a065a4df4d15ad06140aa3af6e88dad74eb6087c43c6632f4333534523 not found: ID does not exist" Jul 02 19:43:09.678967 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.678948 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-dshm\") pod \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " Jul 02 19:43:09.679036 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.678975 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-tmp-dir\") pod \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " Jul 02 19:43:09.679036 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.679016 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-home\") pod \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " Jul 02 19:43:09.679125 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.679052 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-tls-certs\") pod \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " Jul 02 19:43:09.679125 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.679076 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-model-cache\") pod \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " Jul 02 19:43:09.679238 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.679137 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gl8v\" (UniqueName: \"kubernetes.io/projected/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-kube-api-access-8gl8v\") pod \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\" (UID: \"7a8f93e6-97a4-4b67-87c6-801d4f7900c8\") " Jul 02 19:43:09.679294 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.679239 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7a8f93e6-97a4-4b67-87c6-801d4f7900c8" (UID: "7a8f93e6-97a4-4b67-87c6-801d4f7900c8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:43:09.679352 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.679305 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-home" (OuterVolumeSpecName: "home") pod "7a8f93e6-97a4-4b67-87c6-801d4f7900c8" (UID: "7a8f93e6-97a4-4b67-87c6-801d4f7900c8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:43:09.679404 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.679373 2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-tmp-dir\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:43:09.679404 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.679390 2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-home\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:43:09.679477 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.679417 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-model-cache" (OuterVolumeSpecName: "model-cache") pod "7a8f93e6-97a4-4b67-87c6-801d4f7900c8" (UID: "7a8f93e6-97a4-4b67-87c6-801d4f7900c8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:43:09.681212 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.681191 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-dshm" (OuterVolumeSpecName: "dshm") pod "7a8f93e6-97a4-4b67-87c6-801d4f7900c8" (UID: "7a8f93e6-97a4-4b67-87c6-801d4f7900c8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:43:09.681294 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.681216 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7a8f93e6-97a4-4b67-87c6-801d4f7900c8" (UID: "7a8f93e6-97a4-4b67-87c6-801d4f7900c8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:43:09.681294 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.681222 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-kube-api-access-8gl8v" (OuterVolumeSpecName: "kube-api-access-8gl8v") pod "7a8f93e6-97a4-4b67-87c6-801d4f7900c8" (UID: "7a8f93e6-97a4-4b67-87c6-801d4f7900c8"). InnerVolumeSpecName "kube-api-access-8gl8v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:43:09.780295 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.780261 2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-dshm\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:43:09.780295 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.780286 2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-tls-certs\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:43:09.780295 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.780295 2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-model-cache\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:43:09.780295 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.780305 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gl8v\" (UniqueName: \"kubernetes.io/projected/7a8f93e6-97a4-4b67-87c6-801d4f7900c8-kube-api-access-8gl8v\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:43:09.986418 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.986383 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg"] Jul 02 19:43:09.988062 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:09.988040 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-4a101a/auth-test-model-4a101a-kserve-9bdbd5fbc-wslfg"] Jul 02 19:43:10.184835 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.184778 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" podUID="df2fb747-91cc-4c36-a331-59f60e20a2e2" containerName="main" containerID="cri-o://e3002a5d8ff39a15cae10f1aed9d4b025a8829512775dbd9a0a9a2a819e175a6" gracePeriod=45 Jul 02 19:43:10.433406 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.433380 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:43:10.587651 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.587569 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-home\") pod \"df2fb747-91cc-4c36-a331-59f60e20a2e2\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " Jul 02 19:43:10.587651 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.587634 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df2fb747-91cc-4c36-a331-59f60e20a2e2-tls-certs\") pod \"df2fb747-91cc-4c36-a331-59f60e20a2e2\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " Jul 02 19:43:10.587909 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.587660 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-dshm\") pod \"df2fb747-91cc-4c36-a331-59f60e20a2e2\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " Jul 02 19:43:10.587909 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.587722 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-model-cache\") pod \"df2fb747-91cc-4c36-a331-59f60e20a2e2\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " Jul 02 19:43:10.587909 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.587757 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vc98b\" (UniqueName: \"kubernetes.io/projected/df2fb747-91cc-4c36-a331-59f60e20a2e2-kube-api-access-vc98b\") pod \"df2fb747-91cc-4c36-a331-59f60e20a2e2\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " Jul 02 19:43:10.587909 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.587783 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-tmp-dir\") pod \"df2fb747-91cc-4c36-a331-59f60e20a2e2\" (UID: \"df2fb747-91cc-4c36-a331-59f60e20a2e2\") " Jul 02 19:43:10.587909 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.587844 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-home" (OuterVolumeSpecName: "home") pod "df2fb747-91cc-4c36-a331-59f60e20a2e2" (UID: "df2fb747-91cc-4c36-a331-59f60e20a2e2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:43:10.588148 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.588001 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-model-cache" (OuterVolumeSpecName: "model-cache") pod "df2fb747-91cc-4c36-a331-59f60e20a2e2" (UID: "df2fb747-91cc-4c36-a331-59f60e20a2e2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:43:10.588148 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.588019 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "df2fb747-91cc-4c36-a331-59f60e20a2e2" (UID: "df2fb747-91cc-4c36-a331-59f60e20a2e2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:43:10.588148 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.588036 2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-home\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:43:10.589914 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.589890 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-dshm" (OuterVolumeSpecName: "dshm") pod "df2fb747-91cc-4c36-a331-59f60e20a2e2" (UID: "df2fb747-91cc-4c36-a331-59f60e20a2e2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:43:10.590013 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.589914 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df2fb747-91cc-4c36-a331-59f60e20a2e2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "df2fb747-91cc-4c36-a331-59f60e20a2e2" (UID: "df2fb747-91cc-4c36-a331-59f60e20a2e2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:43:10.590053 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.590000 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df2fb747-91cc-4c36-a331-59f60e20a2e2-kube-api-access-vc98b" (OuterVolumeSpecName: "kube-api-access-vc98b") pod "df2fb747-91cc-4c36-a331-59f60e20a2e2" (UID: "df2fb747-91cc-4c36-a331-59f60e20a2e2"). InnerVolumeSpecName "kube-api-access-vc98b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:43:10.669095 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.669058 2566 generic.go:356] "Generic (PLEG): container finished" podID="df2fb747-91cc-4c36-a331-59f60e20a2e2" containerID="e3002a5d8ff39a15cae10f1aed9d4b025a8829512775dbd9a0a9a2a819e175a6" exitCode=0 Jul 02 19:43:10.669274 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.669109 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" Jul 02 19:43:10.669274 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.669145 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" event={"ID":"df2fb747-91cc-4c36-a331-59f60e20a2e2","Type":"ContainerDied","Data":"e3002a5d8ff39a15cae10f1aed9d4b025a8829512775dbd9a0a9a2a819e175a6"} Jul 02 19:43:10.669274 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.669186 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz" event={"ID":"df2fb747-91cc-4c36-a331-59f60e20a2e2","Type":"ContainerDied","Data":"f5cc71bfc80e3c4ebebe20f088a6a9e90cbc5bb9147e861683cdc7d07624ac9c"} Jul 02 19:43:10.669274 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.669199 2566 scope.go:117] "RemoveContainer" containerID="e3002a5d8ff39a15cae10f1aed9d4b025a8829512775dbd9a0a9a2a819e175a6" Jul 02 19:43:10.678184 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.678166 2566 scope.go:117] "RemoveContainer" containerID="e3002a5d8ff39a15cae10f1aed9d4b025a8829512775dbd9a0a9a2a819e175a6" Jul 02 19:43:10.678417 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:43:10.678399 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3002a5d8ff39a15cae10f1aed9d4b025a8829512775dbd9a0a9a2a819e175a6\": container with ID starting with e3002a5d8ff39a15cae10f1aed9d4b025a8829512775dbd9a0a9a2a819e175a6 not found: ID does not exist" containerID="e3002a5d8ff39a15cae10f1aed9d4b025a8829512775dbd9a0a9a2a819e175a6" Jul 02 19:43:10.678464 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.678426 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3002a5d8ff39a15cae10f1aed9d4b025a8829512775dbd9a0a9a2a819e175a6"} err="failed to get container status \"e3002a5d8ff39a15cae10f1aed9d4b025a8829512775dbd9a0a9a2a819e175a6\": rpc error: code = NotFound desc = could not find container \"e3002a5d8ff39a15cae10f1aed9d4b025a8829512775dbd9a0a9a2a819e175a6\": container with ID starting with e3002a5d8ff39a15cae10f1aed9d4b025a8829512775dbd9a0a9a2a819e175a6 not found: ID does not exist" Jul 02 19:43:10.688582 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.688562 2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-dshm\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:43:10.688722 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.688584 2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-model-cache\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:43:10.688722 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.688598 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vc98b\" (UniqueName: \"kubernetes.io/projected/df2fb747-91cc-4c36-a331-59f60e20a2e2-kube-api-access-vc98b\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:43:10.688722 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.688638 2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df2fb747-91cc-4c36-a331-59f60e20a2e2-tmp-dir\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:43:10.688722 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.688655 2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df2fb747-91cc-4c36-a331-59f60e20a2e2-tls-certs\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:43:10.689136 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.689116 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz"] Jul 02 19:43:10.692171 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.692151 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f80343/auth-test-model-f80343-kserve-6ff7544875-z2jpz"] Jul 02 19:43:10.844849 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.844772 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a8f93e6-97a4-4b67-87c6-801d4f7900c8" path="/var/lib/kubelet/pods/7a8f93e6-97a4-4b67-87c6-801d4f7900c8/volumes" Jul 02 19:43:10.845157 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:10.845144 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df2fb747-91cc-4c36-a331-59f60e20a2e2" path="/var/lib/kubelet/pods/df2fb747-91cc-4c36-a331-59f60e20a2e2/volumes" Jul 02 19:43:31.013977 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:31.013944 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:43:32.185729 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.185699 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb"] Jul 02 19:43:32.186118 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.186073 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a8f93e6-97a4-4b67-87c6-801d4f7900c8" containerName="main" Jul 02 19:43:32.186118 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.186086 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a8f93e6-97a4-4b67-87c6-801d4f7900c8" containerName="main" Jul 02 19:43:32.186118 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.186101 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df2fb747-91cc-4c36-a331-59f60e20a2e2" containerName="main" Jul 02 19:43:32.186118 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.186109 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="df2fb747-91cc-4c36-a331-59f60e20a2e2" containerName="main" Jul 02 19:43:32.186253 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.186188 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="df2fb747-91cc-4c36-a331-59f60e20a2e2" containerName="main" Jul 02 19:43:32.186253 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.186203 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a8f93e6-97a4-4b67-87c6-801d4f7900c8" containerName="main" Jul 02 19:43:32.189182 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.189161 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:32.191546 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.191522 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f80343\"/\"openshift-service-ca.crt\"" Jul 02 19:43:32.192532 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.192508 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f80343\"/\"default-dockercfg-ktxdg\"" Jul 02 19:43:32.192532 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.192525 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f80343\"/\"kube-root-ca.crt\"" Jul 02 19:43:32.192626 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.192530 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f80343\"/\"sub-test-model-f80343-kserve-self-signed-certs\"" Jul 02 19:43:32.197898 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.197875 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb"] Jul 02 19:43:32.215066 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.215035 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:43:32.269512 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.269478 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01eee54b-2979-42b8-a1ed-ed4e97386d79-tls-certs\") pod \"sub-test-model-f80343-kserve-5796b47587-f6hlb\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:32.269669 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.269517 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-model-cache\") pod \"sub-test-model-f80343-kserve-5796b47587-f6hlb\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:32.269669 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.269556 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-home\") pod \"sub-test-model-f80343-kserve-5796b47587-f6hlb\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:32.269669 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.269607 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-dshm\") pod \"sub-test-model-f80343-kserve-5796b47587-f6hlb\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:32.269804 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.269700 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvxp9\" (UniqueName: \"kubernetes.io/projected/01eee54b-2979-42b8-a1ed-ed4e97386d79-kube-api-access-bvxp9\") pod \"sub-test-model-f80343-kserve-5796b47587-f6hlb\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:32.269804 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.269731 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-tmp-dir\") pod \"sub-test-model-f80343-kserve-5796b47587-f6hlb\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:32.370446 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.370405 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-tmp-dir\") pod \"sub-test-model-f80343-kserve-5796b47587-f6hlb\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:32.370446 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.370450 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01eee54b-2979-42b8-a1ed-ed4e97386d79-tls-certs\") pod \"sub-test-model-f80343-kserve-5796b47587-f6hlb\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:32.370721 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.370573 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-model-cache\") pod \"sub-test-model-f80343-kserve-5796b47587-f6hlb\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:32.370721 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.370642 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-home\") pod \"sub-test-model-f80343-kserve-5796b47587-f6hlb\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:32.370850 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.370743 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-dshm\") pod \"sub-test-model-f80343-kserve-5796b47587-f6hlb\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:32.370850 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.370815 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvxp9\" (UniqueName: \"kubernetes.io/projected/01eee54b-2979-42b8-a1ed-ed4e97386d79-kube-api-access-bvxp9\") pod \"sub-test-model-f80343-kserve-5796b47587-f6hlb\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:32.370958 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.370874 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-tmp-dir\") pod \"sub-test-model-f80343-kserve-5796b47587-f6hlb\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:32.370958 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.370917 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-model-cache\") pod \"sub-test-model-f80343-kserve-5796b47587-f6hlb\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:32.371056 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.370959 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-home\") pod \"sub-test-model-f80343-kserve-5796b47587-f6hlb\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:32.372797 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.372772 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-dshm\") pod \"sub-test-model-f80343-kserve-5796b47587-f6hlb\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:32.373090 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.373071 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01eee54b-2979-42b8-a1ed-ed4e97386d79-tls-certs\") pod \"sub-test-model-f80343-kserve-5796b47587-f6hlb\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:32.378800 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.378777 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvxp9\" (UniqueName: \"kubernetes.io/projected/01eee54b-2979-42b8-a1ed-ed4e97386d79-kube-api-access-bvxp9\") pod \"sub-test-model-f80343-kserve-5796b47587-f6hlb\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:32.498719 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.498662 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:32.633860 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.633832 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb"] Jul 02 19:43:32.635987 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:43:32.635951 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01eee54b_2979_42b8_a1ed_ed4e97386d79.slice/crio-14193e5c889b0969308f039d1ceab24a8a322f156a128c2221cac44b5152a77c WatchSource:0}: Error finding container 14193e5c889b0969308f039d1ceab24a8a322f156a128c2221cac44b5152a77c: Status 404 returned error can't find the container with id 14193e5c889b0969308f039d1ceab24a8a322f156a128c2221cac44b5152a77c Jul 02 19:43:32.783938 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.783849 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" event={"ID":"01eee54b-2979-42b8-a1ed-ed4e97386d79","Type":"ContainerStarted","Data":"25d89638b371378a1f4cdf5e6bbe2df91bd9c4e1cb1e7b875bb9dfa2233dfd2f"} Jul 02 19:43:32.783938 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.783899 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" event={"ID":"01eee54b-2979-42b8-a1ed-ed4e97386d79","Type":"ContainerStarted","Data":"14193e5c889b0969308f039d1ceab24a8a322f156a128c2221cac44b5152a77c"} Jul 02 19:43:32.802795 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:32.802736 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" podStartSLOduration=0.802717322 podStartE2EDuration="802.717322ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:43:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:43:32.800517429 +0000 UTC m=+2292.727526142" watchObservedRunningTime="2026-07-02 19:43:32.802717322 +0000 UTC m=+2292.729726037" Jul 02 19:43:42.499414 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:42.499373 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:42.499414 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:42.499428 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:42.512094 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:42.512065 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:43:42.848181 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:43:42.848091 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:44:17.006042 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:17.005955 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:44:17.851569 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:17.851539 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb"] Jul 02 19:44:17.884794 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:17.884761 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:44:22.782489 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.782453 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v"] Jul 02 19:44:22.786024 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.786007 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:22.788574 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.788550 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4a101a\"/\"kube-root-ca.crt\"" Jul 02 19:44:22.788703 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.788645 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4a101a\"/\"openshift-service-ca.crt\"" Jul 02 19:44:22.789744 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.789722 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4a101a\"/\"rate-test-model-4a101a-kserve-self-signed-certs\"" Jul 02 19:44:22.789867 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.789753 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4a101a\"/\"default-dockercfg-nbl6m\"" Jul 02 19:44:22.796292 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.796269 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v"] Jul 02 19:44:22.820234 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.820200 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-tmp-dir\") pod \"rate-test-model-4a101a-kserve-745f5f48ff-n5g4v\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:22.820383 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.820270 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-model-cache\") pod \"rate-test-model-4a101a-kserve-745f5f48ff-n5g4v\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:22.820383 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.820287 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjhss\" (UniqueName: \"kubernetes.io/projected/39604f1f-5697-4b2d-b796-e2961ea8be5d-kube-api-access-qjhss\") pod \"rate-test-model-4a101a-kserve-745f5f48ff-n5g4v\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:22.820383 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.820340 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-dshm\") pod \"rate-test-model-4a101a-kserve-745f5f48ff-n5g4v\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:22.820507 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.820402 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39604f1f-5697-4b2d-b796-e2961ea8be5d-tls-certs\") pod \"rate-test-model-4a101a-kserve-745f5f48ff-n5g4v\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:22.820507 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.820439 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-home\") pod \"rate-test-model-4a101a-kserve-745f5f48ff-n5g4v\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:22.836916 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.836889 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:44:22.921513 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.921472 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-dshm\") pod \"rate-test-model-4a101a-kserve-745f5f48ff-n5g4v\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:22.921729 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.921519 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39604f1f-5697-4b2d-b796-e2961ea8be5d-tls-certs\") pod \"rate-test-model-4a101a-kserve-745f5f48ff-n5g4v\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:22.921729 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.921571 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-home\") pod \"rate-test-model-4a101a-kserve-745f5f48ff-n5g4v\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:22.921729 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.921715 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-tmp-dir\") pod \"rate-test-model-4a101a-kserve-745f5f48ff-n5g4v\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:22.921948 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.921922 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-model-cache\") pod \"rate-test-model-4a101a-kserve-745f5f48ff-n5g4v\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:22.922018 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.921966 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjhss\" (UniqueName: \"kubernetes.io/projected/39604f1f-5697-4b2d-b796-e2961ea8be5d-kube-api-access-qjhss\") pod \"rate-test-model-4a101a-kserve-745f5f48ff-n5g4v\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:22.922409 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.922379 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-model-cache\") pod \"rate-test-model-4a101a-kserve-745f5f48ff-n5g4v\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:22.922544 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.922415 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-tmp-dir\") pod \"rate-test-model-4a101a-kserve-745f5f48ff-n5g4v\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:22.922544 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.922449 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-home\") pod \"rate-test-model-4a101a-kserve-745f5f48ff-n5g4v\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:22.923986 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.923962 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-dshm\") pod \"rate-test-model-4a101a-kserve-745f5f48ff-n5g4v\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:22.924251 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.924233 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39604f1f-5697-4b2d-b796-e2961ea8be5d-tls-certs\") pod \"rate-test-model-4a101a-kserve-745f5f48ff-n5g4v\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:22.930211 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:22.930188 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjhss\" (UniqueName: \"kubernetes.io/projected/39604f1f-5697-4b2d-b796-e2961ea8be5d-kube-api-access-qjhss\") pod \"rate-test-model-4a101a-kserve-745f5f48ff-n5g4v\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:23.096180 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:23.096077 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:23.233716 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:23.233686 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v"] Jul 02 19:44:23.235406 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:44:23.235380 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39604f1f_5697_4b2d_b796_e2961ea8be5d.slice/crio-bd1a2859f1439d914031ce1caae6827c858f1fc8384559110c5cf88269f57277 WatchSource:0}: Error finding container bd1a2859f1439d914031ce1caae6827c858f1fc8384559110c5cf88269f57277: Status 404 returned error can't find the container with id bd1a2859f1439d914031ce1caae6827c858f1fc8384559110c5cf88269f57277 Jul 02 19:44:24.032898 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:24.032863 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" event={"ID":"39604f1f-5697-4b2d-b796-e2961ea8be5d","Type":"ContainerStarted","Data":"29bb0ad5763b553313d008d3c8e9adf30f98219dea744e594c04da64628e0a00"} Jul 02 19:44:24.032898 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:24.032906 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" event={"ID":"39604f1f-5697-4b2d-b796-e2961ea8be5d","Type":"ContainerStarted","Data":"bd1a2859f1439d914031ce1caae6827c858f1fc8384559110c5cf88269f57277"} Jul 02 19:44:24.058747 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:24.058658 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" podStartSLOduration=2.058644608 podStartE2EDuration="2.058644608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:44:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:44:24.057892974 +0000 UTC m=+2343.984901686" watchObservedRunningTime="2026-07-02 19:44:24.058644608 +0000 UTC m=+2343.985653320" Jul 02 19:44:28.463872 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:28.463834 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:44:28.621690 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:28.621645 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:44:30.945287 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:30.945250 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:44:32.871044 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:32.871008 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" podUID="01eee54b-2979-42b8-a1ed-ed4e97386d79" containerName="main" containerID="cri-o://25d89638b371378a1f4cdf5e6bbe2df91bd9c4e1cb1e7b875bb9dfa2233dfd2f" gracePeriod=45 Jul 02 19:44:33.085982 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.085948 2566 generic.go:356] "Generic (PLEG): container finished" podID="01eee54b-2979-42b8-a1ed-ed4e97386d79" containerID="25d89638b371378a1f4cdf5e6bbe2df91bd9c4e1cb1e7b875bb9dfa2233dfd2f" exitCode=0 Jul 02 19:44:33.086157 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.086022 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" event={"ID":"01eee54b-2979-42b8-a1ed-ed4e97386d79","Type":"ContainerDied","Data":"25d89638b371378a1f4cdf5e6bbe2df91bd9c4e1cb1e7b875bb9dfa2233dfd2f"} Jul 02 19:44:33.096172 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.096150 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:33.096258 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.096193 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:33.109047 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.109026 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:33.121284 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.121235 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:44:33.220279 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.220243 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-tmp-dir\") pod \"01eee54b-2979-42b8-a1ed-ed4e97386d79\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " Jul 02 19:44:33.220461 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.220296 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01eee54b-2979-42b8-a1ed-ed4e97386d79-tls-certs\") pod \"01eee54b-2979-42b8-a1ed-ed4e97386d79\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " Jul 02 19:44:33.220461 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.220353 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-dshm\") pod \"01eee54b-2979-42b8-a1ed-ed4e97386d79\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " Jul 02 19:44:33.220461 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.220394 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-model-cache\") pod \"01eee54b-2979-42b8-a1ed-ed4e97386d79\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " Jul 02 19:44:33.220611 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.220547 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-home\") pod \"01eee54b-2979-42b8-a1ed-ed4e97386d79\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " Jul 02 19:44:33.220611 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.220557 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "01eee54b-2979-42b8-a1ed-ed4e97386d79" (UID: "01eee54b-2979-42b8-a1ed-ed4e97386d79"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:44:33.220611 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.220600 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvxp9\" (UniqueName: \"kubernetes.io/projected/01eee54b-2979-42b8-a1ed-ed4e97386d79-kube-api-access-bvxp9\") pod \"01eee54b-2979-42b8-a1ed-ed4e97386d79\" (UID: \"01eee54b-2979-42b8-a1ed-ed4e97386d79\") " Jul 02 19:44:33.220783 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.220707 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-model-cache" (OuterVolumeSpecName: "model-cache") pod "01eee54b-2979-42b8-a1ed-ed4e97386d79" (UID: "01eee54b-2979-42b8-a1ed-ed4e97386d79"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:44:33.220848 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.220796 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-home" (OuterVolumeSpecName: "home") pod "01eee54b-2979-42b8-a1ed-ed4e97386d79" (UID: "01eee54b-2979-42b8-a1ed-ed4e97386d79"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:44:33.221000 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.220979 2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-tmp-dir\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:44:33.221000 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.220997 2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-model-cache\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:44:33.221000 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.221008 2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-home\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:44:33.222646 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.222619 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01eee54b-2979-42b8-a1ed-ed4e97386d79-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "01eee54b-2979-42b8-a1ed-ed4e97386d79" (UID: "01eee54b-2979-42b8-a1ed-ed4e97386d79"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:44:33.222765 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.222670 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-dshm" (OuterVolumeSpecName: "dshm") pod "01eee54b-2979-42b8-a1ed-ed4e97386d79" (UID: "01eee54b-2979-42b8-a1ed-ed4e97386d79"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:44:33.222765 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.222743 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01eee54b-2979-42b8-a1ed-ed4e97386d79-kube-api-access-bvxp9" (OuterVolumeSpecName: "kube-api-access-bvxp9") pod "01eee54b-2979-42b8-a1ed-ed4e97386d79" (UID: "01eee54b-2979-42b8-a1ed-ed4e97386d79"). InnerVolumeSpecName "kube-api-access-bvxp9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:44:33.322137 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.322100 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvxp9\" (UniqueName: \"kubernetes.io/projected/01eee54b-2979-42b8-a1ed-ed4e97386d79-kube-api-access-bvxp9\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:44:33.322137 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.322131 2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01eee54b-2979-42b8-a1ed-ed4e97386d79-tls-certs\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:44:33.322137 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:33.322146 2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01eee54b-2979-42b8-a1ed-ed4e97386d79-dshm\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:44:34.092806 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:34.092775 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" Jul 02 19:44:34.093331 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:34.092821 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb" event={"ID":"01eee54b-2979-42b8-a1ed-ed4e97386d79","Type":"ContainerDied","Data":"14193e5c889b0969308f039d1ceab24a8a322f156a128c2221cac44b5152a77c"} Jul 02 19:44:34.093331 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:34.092871 2566 scope.go:117] "RemoveContainer" containerID="25d89638b371378a1f4cdf5e6bbe2df91bd9c4e1cb1e7b875bb9dfa2233dfd2f" Jul 02 19:44:34.108038 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:34.108016 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:44:34.119776 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:34.119754 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb"] Jul 02 19:44:34.124270 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:34.124249 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f80343/sub-test-model-f80343-kserve-5796b47587-f6hlb"] Jul 02 19:44:34.843523 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:34.843491 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01eee54b-2979-42b8-a1ed-ed4e97386d79" path="/var/lib/kubelet/pods/01eee54b-2979-42b8-a1ed-ed4e97386d79/volumes" Jul 02 19:44:43.389547 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.389513 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr"] Jul 02 19:44:43.390075 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.390044 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01eee54b-2979-42b8-a1ed-ed4e97386d79" containerName="main" Jul 02 19:44:43.390156 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.390079 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="01eee54b-2979-42b8-a1ed-ed4e97386d79" containerName="main" Jul 02 19:44:43.390213 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.390195 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="01eee54b-2979-42b8-a1ed-ed4e97386d79" containerName="main" Jul 02 19:44:43.394719 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.394697 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:43.397367 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.397348 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f80343\"/\"openshift-service-ca.crt\"" Jul 02 19:44:43.398324 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.398307 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f80343\"/\"default-dockercfg-ktxdg\"" Jul 02 19:44:43.398621 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.398600 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f80343\"/\"kube-root-ca.crt\"" Jul 02 19:44:43.398756 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.398740 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f80343\"/\"rate-test-model-f80343-kserve-self-signed-certs\"" Jul 02 19:44:43.408751 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.408728 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr"] Jul 02 19:44:43.454991 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.454963 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:44:43.512005 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.511972 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-home\") pod \"rate-test-model-f80343-kserve-69b9bd74cf-bk4cr\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:43.512005 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.512003 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-dshm\") pod \"rate-test-model-f80343-kserve-69b9bd74cf-bk4cr\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:43.512215 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.512101 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d718c32c-88f5-45cc-9309-92624d5a0181-tls-certs\") pod \"rate-test-model-f80343-kserve-69b9bd74cf-bk4cr\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:43.512215 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.512148 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-tmp-dir\") pod \"rate-test-model-f80343-kserve-69b9bd74cf-bk4cr\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:43.512215 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.512171 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-model-cache\") pod \"rate-test-model-f80343-kserve-69b9bd74cf-bk4cr\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:43.512215 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.512190 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvzrn\" (UniqueName: \"kubernetes.io/projected/d718c32c-88f5-45cc-9309-92624d5a0181-kube-api-access-lvzrn\") pod \"rate-test-model-f80343-kserve-69b9bd74cf-bk4cr\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:43.613406 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.613370 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d718c32c-88f5-45cc-9309-92624d5a0181-tls-certs\") pod \"rate-test-model-f80343-kserve-69b9bd74cf-bk4cr\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:43.613606 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.613418 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-tmp-dir\") pod \"rate-test-model-f80343-kserve-69b9bd74cf-bk4cr\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:43.613606 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.613449 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-model-cache\") pod \"rate-test-model-f80343-kserve-69b9bd74cf-bk4cr\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:43.613606 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.613476 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvzrn\" (UniqueName: \"kubernetes.io/projected/d718c32c-88f5-45cc-9309-92624d5a0181-kube-api-access-lvzrn\") pod \"rate-test-model-f80343-kserve-69b9bd74cf-bk4cr\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:43.613606 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.613541 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-home\") pod \"rate-test-model-f80343-kserve-69b9bd74cf-bk4cr\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:43.613606 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.613564 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-dshm\") pod \"rate-test-model-f80343-kserve-69b9bd74cf-bk4cr\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:43.613963 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.613942 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-tmp-dir\") pod \"rate-test-model-f80343-kserve-69b9bd74cf-bk4cr\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:43.614034 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.613999 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-model-cache\") pod \"rate-test-model-f80343-kserve-69b9bd74cf-bk4cr\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:43.614102 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.614037 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-home\") pod \"rate-test-model-f80343-kserve-69b9bd74cf-bk4cr\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:43.615913 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.615883 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-dshm\") pod \"rate-test-model-f80343-kserve-69b9bd74cf-bk4cr\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:43.616021 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.615981 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d718c32c-88f5-45cc-9309-92624d5a0181-tls-certs\") pod \"rate-test-model-f80343-kserve-69b9bd74cf-bk4cr\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:43.622202 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.622178 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvzrn\" (UniqueName: \"kubernetes.io/projected/d718c32c-88f5-45cc-9309-92624d5a0181-kube-api-access-lvzrn\") pod \"rate-test-model-f80343-kserve-69b9bd74cf-bk4cr\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:43.703917 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.703824 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:43.841827 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:43.841803 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr"] Jul 02 19:44:43.843779 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:44:43.843749 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd718c32c_88f5_45cc_9309_92624d5a0181.slice/crio-ae6a2b0278c8427af11827b431256087ab64db27533efa1e8db6206760fa8a01 WatchSource:0}: Error finding container ae6a2b0278c8427af11827b431256087ab64db27533efa1e8db6206760fa8a01: Status 404 returned error can't find the container with id ae6a2b0278c8427af11827b431256087ab64db27533efa1e8db6206760fa8a01 Jul 02 19:44:44.146092 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:44.146055 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" event={"ID":"d718c32c-88f5-45cc-9309-92624d5a0181","Type":"ContainerStarted","Data":"5e75852d901657eb48c1fc7b9363a54e17efbb7f1c201d01dd667a4f29a882a4"} Jul 02 19:44:44.146307 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:44.146104 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" event={"ID":"d718c32c-88f5-45cc-9309-92624d5a0181","Type":"ContainerStarted","Data":"ae6a2b0278c8427af11827b431256087ab64db27533efa1e8db6206760fa8a01"} Jul 02 19:44:44.163827 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:44.163777 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" podStartSLOduration=1.163761474 podStartE2EDuration="1.163761474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 19:44:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:44:44.161577092 +0000 UTC m=+2364.088585799" watchObservedRunningTime="2026-07-02 19:44:44.163761474 +0000 UTC m=+2364.090770202" Jul 02 19:44:53.704926 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:53.704891 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:53.705443 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:53.704945 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:53.717736 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:53.717710 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:44:54.212820 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:44:54.212788 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:45:04.084606 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:04.084563 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:45:04.181134 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:04.181102 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:45:12.857903 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:12.857856 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:45:12.915873 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:12.915846 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:45:37.112913 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:37.112878 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v"] Jul 02 19:45:37.143286 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:37.143259 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:45:37.989739 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:37.988843 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr"] Jul 02 19:45:38.007279 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:38.007246 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:45:39.181414 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:39.181381 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:45:39.595223 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:39.595193 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:45:40.079822 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:40.079788 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:45:40.138315 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:40.138283 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:45:44.746875 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:44.746800 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:45:52.132581 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.132532 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" podUID="39604f1f-5697-4b2d-b796-e2961ea8be5d" containerName="main" containerID="cri-o://29bb0ad5763b553313d008d3c8e9adf30f98219dea744e594c04da64628e0a00" gracePeriod=45 Jul 02 19:45:52.379327 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.379296 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:45:52.433388 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.433307 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-tmp-dir\") pod \"39604f1f-5697-4b2d-b796-e2961ea8be5d\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " Jul 02 19:45:52.433388 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.433381 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-dshm\") pod \"39604f1f-5697-4b2d-b796-e2961ea8be5d\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " Jul 02 19:45:52.433622 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.433417 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-model-cache\") pod \"39604f1f-5697-4b2d-b796-e2961ea8be5d\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " Jul 02 19:45:52.433622 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.433441 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39604f1f-5697-4b2d-b796-e2961ea8be5d-tls-certs\") pod \"39604f1f-5697-4b2d-b796-e2961ea8be5d\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " Jul 02 19:45:52.433622 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.433476 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjhss\" (UniqueName: \"kubernetes.io/projected/39604f1f-5697-4b2d-b796-e2961ea8be5d-kube-api-access-qjhss\") pod \"39604f1f-5697-4b2d-b796-e2961ea8be5d\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " Jul 02 19:45:52.433622 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.433502 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-home\") pod \"39604f1f-5697-4b2d-b796-e2961ea8be5d\" (UID: \"39604f1f-5697-4b2d-b796-e2961ea8be5d\") " Jul 02 19:45:52.433622 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.433591 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "39604f1f-5697-4b2d-b796-e2961ea8be5d" (UID: "39604f1f-5697-4b2d-b796-e2961ea8be5d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:45:52.433911 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.433739 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-model-cache" (OuterVolumeSpecName: "model-cache") pod "39604f1f-5697-4b2d-b796-e2961ea8be5d" (UID: "39604f1f-5697-4b2d-b796-e2961ea8be5d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:45:52.433911 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.433865 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-home" (OuterVolumeSpecName: "home") pod "39604f1f-5697-4b2d-b796-e2961ea8be5d" (UID: "39604f1f-5697-4b2d-b796-e2961ea8be5d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:45:52.433911 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.433890 2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-model-cache\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:45:52.433911 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.433910 2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-tmp-dir\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:45:52.435486 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.435461 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-dshm" (OuterVolumeSpecName: "dshm") pod "39604f1f-5697-4b2d-b796-e2961ea8be5d" (UID: "39604f1f-5697-4b2d-b796-e2961ea8be5d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:45:52.435637 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.435621 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39604f1f-5697-4b2d-b796-e2961ea8be5d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "39604f1f-5697-4b2d-b796-e2961ea8be5d" (UID: "39604f1f-5697-4b2d-b796-e2961ea8be5d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:45:52.435709 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.435659 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39604f1f-5697-4b2d-b796-e2961ea8be5d-kube-api-access-qjhss" (OuterVolumeSpecName: "kube-api-access-qjhss") pod "39604f1f-5697-4b2d-b796-e2961ea8be5d" (UID: "39604f1f-5697-4b2d-b796-e2961ea8be5d"). InnerVolumeSpecName "kube-api-access-qjhss". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:45:52.480853 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.480817 2566 generic.go:356] "Generic (PLEG): container finished" podID="39604f1f-5697-4b2d-b796-e2961ea8be5d" containerID="29bb0ad5763b553313d008d3c8e9adf30f98219dea744e594c04da64628e0a00" exitCode=0 Jul 02 19:45:52.481018 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.480864 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" Jul 02 19:45:52.481018 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.480901 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" event={"ID":"39604f1f-5697-4b2d-b796-e2961ea8be5d","Type":"ContainerDied","Data":"29bb0ad5763b553313d008d3c8e9adf30f98219dea744e594c04da64628e0a00"} Jul 02 19:45:52.481018 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.480949 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v" event={"ID":"39604f1f-5697-4b2d-b796-e2961ea8be5d","Type":"ContainerDied","Data":"bd1a2859f1439d914031ce1caae6827c858f1fc8384559110c5cf88269f57277"} Jul 02 19:45:52.481018 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.480967 2566 scope.go:117] "RemoveContainer" containerID="29bb0ad5763b553313d008d3c8e9adf30f98219dea744e594c04da64628e0a00" Jul 02 19:45:52.489682 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.489653 2566 scope.go:117] "RemoveContainer" containerID="29bb0ad5763b553313d008d3c8e9adf30f98219dea744e594c04da64628e0a00" Jul 02 19:45:52.489938 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:45:52.489920 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29bb0ad5763b553313d008d3c8e9adf30f98219dea744e594c04da64628e0a00\": container with ID starting with 29bb0ad5763b553313d008d3c8e9adf30f98219dea744e594c04da64628e0a00 not found: ID does not exist" containerID="29bb0ad5763b553313d008d3c8e9adf30f98219dea744e594c04da64628e0a00" Jul 02 19:45:52.490014 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.489945 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29bb0ad5763b553313d008d3c8e9adf30f98219dea744e594c04da64628e0a00"} err="failed to get container status \"29bb0ad5763b553313d008d3c8e9adf30f98219dea744e594c04da64628e0a00\": rpc error: code = NotFound desc = could not find container \"29bb0ad5763b553313d008d3c8e9adf30f98219dea744e594c04da64628e0a00\": container with ID starting with 29bb0ad5763b553313d008d3c8e9adf30f98219dea744e594c04da64628e0a00 not found: ID does not exist" Jul 02 19:45:52.501624 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.501595 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v"] Jul 02 19:45:52.507533 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.507504 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-4a101a/rate-test-model-4a101a-kserve-745f5f48ff-n5g4v"] Jul 02 19:45:52.534655 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.534631 2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-dshm\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:45:52.534655 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.534654 2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39604f1f-5697-4b2d-b796-e2961ea8be5d-tls-certs\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:45:52.534826 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.534664 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjhss\" (UniqueName: \"kubernetes.io/projected/39604f1f-5697-4b2d-b796-e2961ea8be5d-kube-api-access-qjhss\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:45:52.534826 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.534692 2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39604f1f-5697-4b2d-b796-e2961ea8be5d-home\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:45:52.843957 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:52.843924 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39604f1f-5697-4b2d-b796-e2961ea8be5d" path="/var/lib/kubelet/pods/39604f1f-5697-4b2d-b796-e2961ea8be5d/volumes" Jul 02 19:45:53.012372 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.012335 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" podUID="d718c32c-88f5-45cc-9309-92624d5a0181" containerName="main" containerID="cri-o://5e75852d901657eb48c1fc7b9363a54e17efbb7f1c201d01dd667a4f29a882a4" gracePeriod=45 Jul 02 19:45:53.258985 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.258956 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:45:53.342078 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.342038 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-tmp-dir\") pod \"d718c32c-88f5-45cc-9309-92624d5a0181\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " Jul 02 19:45:53.342292 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.342103 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-model-cache\") pod \"d718c32c-88f5-45cc-9309-92624d5a0181\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " Jul 02 19:45:53.342292 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.342149 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvzrn\" (UniqueName: \"kubernetes.io/projected/d718c32c-88f5-45cc-9309-92624d5a0181-kube-api-access-lvzrn\") pod \"d718c32c-88f5-45cc-9309-92624d5a0181\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " Jul 02 19:45:53.342292 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.342180 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-dshm\") pod \"d718c32c-88f5-45cc-9309-92624d5a0181\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " Jul 02 19:45:53.342292 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.342228 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-home\") pod \"d718c32c-88f5-45cc-9309-92624d5a0181\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " Jul 02 19:45:53.342292 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.342249 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d718c32c-88f5-45cc-9309-92624d5a0181-tls-certs\") pod \"d718c32c-88f5-45cc-9309-92624d5a0181\" (UID: \"d718c32c-88f5-45cc-9309-92624d5a0181\") " Jul 02 19:45:53.342538 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.342351 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d718c32c-88f5-45cc-9309-92624d5a0181" (UID: "d718c32c-88f5-45cc-9309-92624d5a0181"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:45:53.342538 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.342377 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-model-cache" (OuterVolumeSpecName: "model-cache") pod "d718c32c-88f5-45cc-9309-92624d5a0181" (UID: "d718c32c-88f5-45cc-9309-92624d5a0181"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:45:53.342538 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.342521 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-home" (OuterVolumeSpecName: "home") pod "d718c32c-88f5-45cc-9309-92624d5a0181" (UID: "d718c32c-88f5-45cc-9309-92624d5a0181"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:45:53.342715 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.342616 2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-model-cache\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:45:53.342715 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.342635 2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-home\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:45:53.342715 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.342646 2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-tmp-dir\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:45:53.344329 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.344301 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d718c32c-88f5-45cc-9309-92624d5a0181-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d718c32c-88f5-45cc-9309-92624d5a0181" (UID: "d718c32c-88f5-45cc-9309-92624d5a0181"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 19:45:53.344329 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.344306 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-dshm" (OuterVolumeSpecName: "dshm") pod "d718c32c-88f5-45cc-9309-92624d5a0181" (UID: "d718c32c-88f5-45cc-9309-92624d5a0181"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 19:45:53.344491 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.344340 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d718c32c-88f5-45cc-9309-92624d5a0181-kube-api-access-lvzrn" (OuterVolumeSpecName: "kube-api-access-lvzrn") pod "d718c32c-88f5-45cc-9309-92624d5a0181" (UID: "d718c32c-88f5-45cc-9309-92624d5a0181"). InnerVolumeSpecName "kube-api-access-lvzrn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 19:45:53.444089 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.444001 2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d718c32c-88f5-45cc-9309-92624d5a0181-tls-certs\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:45:53.444089 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.444030 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvzrn\" (UniqueName: \"kubernetes.io/projected/d718c32c-88f5-45cc-9309-92624d5a0181-kube-api-access-lvzrn\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:45:53.444089 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.444039 2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d718c32c-88f5-45cc-9309-92624d5a0181-dshm\") on node \"ip-10-0-143-205.ec2.internal\" DevicePath \"\"" Jul 02 19:45:53.488939 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.488904 2566 generic.go:356] "Generic (PLEG): container finished" podID="d718c32c-88f5-45cc-9309-92624d5a0181" containerID="5e75852d901657eb48c1fc7b9363a54e17efbb7f1c201d01dd667a4f29a882a4" exitCode=0 Jul 02 19:45:53.489114 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.488947 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" Jul 02 19:45:53.489114 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.488965 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" event={"ID":"d718c32c-88f5-45cc-9309-92624d5a0181","Type":"ContainerDied","Data":"5e75852d901657eb48c1fc7b9363a54e17efbb7f1c201d01dd667a4f29a882a4"} Jul 02 19:45:53.489114 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.488996 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr" event={"ID":"d718c32c-88f5-45cc-9309-92624d5a0181","Type":"ContainerDied","Data":"ae6a2b0278c8427af11827b431256087ab64db27533efa1e8db6206760fa8a01"} Jul 02 19:45:53.489114 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.489013 2566 scope.go:117] "RemoveContainer" containerID="5e75852d901657eb48c1fc7b9363a54e17efbb7f1c201d01dd667a4f29a882a4" Jul 02 19:45:53.500105 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.500086 2566 scope.go:117] "RemoveContainer" containerID="5e75852d901657eb48c1fc7b9363a54e17efbb7f1c201d01dd667a4f29a882a4" Jul 02 19:45:53.500390 ip-10-0-143-205 kubenswrapper[2566]: E0702 19:45:53.500369 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e75852d901657eb48c1fc7b9363a54e17efbb7f1c201d01dd667a4f29a882a4\": container with ID starting with 5e75852d901657eb48c1fc7b9363a54e17efbb7f1c201d01dd667a4f29a882a4 not found: ID does not exist" containerID="5e75852d901657eb48c1fc7b9363a54e17efbb7f1c201d01dd667a4f29a882a4" Jul 02 19:45:53.500440 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.500398 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e75852d901657eb48c1fc7b9363a54e17efbb7f1c201d01dd667a4f29a882a4"} err="failed to get container status \"5e75852d901657eb48c1fc7b9363a54e17efbb7f1c201d01dd667a4f29a882a4\": rpc error: code = NotFound desc = could not find container \"5e75852d901657eb48c1fc7b9363a54e17efbb7f1c201d01dd667a4f29a882a4\": container with ID starting with 5e75852d901657eb48c1fc7b9363a54e17efbb7f1c201d01dd667a4f29a882a4 not found: ID does not exist" Jul 02 19:45:53.512927 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.512904 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr"] Jul 02 19:45:53.516163 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:53.516141 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f80343/rate-test-model-f80343-kserve-69b9bd74cf-bk4cr"] Jul 02 19:45:54.848839 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:45:54.848806 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d718c32c-88f5-45cc-9309-92624d5a0181" path="/var/lib/kubelet/pods/d718c32c-88f5-45cc-9309-92624d5a0181/volumes" Jul 02 19:46:20.066696 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:20.066642 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:46:21.514145 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:21.514111 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:46:21.786300 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:21.786191 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:46:22.024859 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:22.024823 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:46:22.208669 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:22.208580 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:46:22.594428 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:22.594394 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:46:22.694904 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:22.694872 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:46:22.869298 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:22.869219 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:46:28.663224 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:28.663190 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:46:28.731949 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:28.731911 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:46:28.924686 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:28.924596 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:46:29.148958 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:29.148930 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:46:29.556323 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:29.556291 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-59c4bf44fc-h2bfx_0a1736aa-2aed-4664-9891-1ad629369467/authorino/0.log" Jul 02 19:46:31.877761 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:31.877704 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:46:35.363243 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:35.363215 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cf784d988-ghsw5_24127ebd-5316-48db-b150-026731d58a15/copy-manifests/0.log" Jul 02 19:46:35.377093 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:35.377057 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cf784d988-ghsw5_24127ebd-5316-48db-b150-026731d58a15/manager/0.log" Jul 02 19:46:36.251994 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:36.251961 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7_4cfcc67b-796c-4fa4-9d84-ad9c8daecd28/util/0.log" Jul 02 19:46:36.264577 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:36.264534 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7_4cfcc67b-796c-4fa4-9d84-ad9c8daecd28/pull/0.log" Jul 02 19:46:36.278642 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:36.278606 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7_4cfcc67b-796c-4fa4-9d84-ad9c8daecd28/extract/0.log" Jul 02 19:46:36.779643 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:36.772834 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-59c4bf44fc-h2bfx_0a1736aa-2aed-4664-9891-1ad629369467/authorino/0.log" Jul 02 19:46:37.350569 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:37.350536 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-r62zd_7e58977e-8f80-41e2-b58c-76cc244d58f9/manager/0.log" Jul 02 19:46:38.170732 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:38.170703 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c665485c8-s2bzz_2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74/kube-auth-proxy/0.log" Jul 02 19:46:39.204332 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:39.204302 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-c72fp_4ff052e2-e15b-4e4e-871f-1f4c943d9498/main/0.log" Jul 02 19:46:39.429478 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:39.429447 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db4bdnx_85f4eefb-e21a-4bc5-9d05-532dd0a2328c/main/0.log" Jul 02 19:46:39.669771 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:39.669739 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-qvch9_c84a8d95-1ddf-4d7e-b875-693224a0e3c0/main/0.log" Jul 02 19:46:43.363833 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:43.363797 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:46:43.437098 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:43.437060 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" limit=1024 Jul 02 19:46:46.253467 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:46.253426 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-txtgt_0e7a0e42-4fee-4788-ba6a-4b7de6ed04ef/global-pull-secret-syncer/0.log" Jul 02 19:46:46.305903 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:46.305871 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bl8kt_a61e4596-f5a1-439d-a1e5-909708c4a2a0/konnectivity-agent/0.log" Jul 02 19:46:46.455974 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:46.455942 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-205.ec2.internal_443b95f9a11c785c66a6608d9d6fba98/haproxy/0.log" Jul 02 19:46:51.019057 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:51.019025 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7_4cfcc67b-796c-4fa4-9d84-ad9c8daecd28/extract/0.log" Jul 02 19:46:51.046988 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:51.046956 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7_4cfcc67b-796c-4fa4-9d84-ad9c8daecd28/util/0.log" Jul 02 19:46:51.080398 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:51.080371 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599ncd7_4cfcc67b-796c-4fa4-9d84-ad9c8daecd28/pull/0.log" Jul 02 19:46:51.623329 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:51.623291 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-59c4bf44fc-h2bfx_0a1736aa-2aed-4664-9891-1ad629369467/authorino/0.log" Jul 02 19:46:51.881907 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:51.881799 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-r62zd_7e58977e-8f80-41e2-b58c-76cc244d58f9/manager/0.log" Jul 02 19:46:53.685044 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:53.684961 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6cc8242f-eb33-4f83-9cfc-c3b069dc72ac/alertmanager/0.log" Jul 02 19:46:53.707931 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:53.707909 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6cc8242f-eb33-4f83-9cfc-c3b069dc72ac/config-reloader/0.log" Jul 02 19:46:53.730567 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:53.730545 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6cc8242f-eb33-4f83-9cfc-c3b069dc72ac/kube-rbac-proxy-web/0.log" Jul 02 19:46:53.756304 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:53.756278 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6cc8242f-eb33-4f83-9cfc-c3b069dc72ac/kube-rbac-proxy/0.log" Jul 02 19:46:53.778732 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:53.778694 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6cc8242f-eb33-4f83-9cfc-c3b069dc72ac/kube-rbac-proxy-metric/0.log" Jul 02 19:46:53.806047 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:53.806018 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6cc8242f-eb33-4f83-9cfc-c3b069dc72ac/prom-label-proxy/0.log" Jul 02 19:46:53.828271 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:53.828248 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6cc8242f-eb33-4f83-9cfc-c3b069dc72ac/init-config-reloader/0.log" Jul 02 19:46:54.042616 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:54.042582 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-hrhd2_e09cfa31-6ce1-485a-9539-1817701d1e95/monitoring-plugin/0.log" Jul 02 19:46:54.237918 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:54.233633 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hskfk_a4adc67b-70b1-40cb-83f2-1d01aa96683e/node-exporter/0.log" Jul 02 19:46:54.256523 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:54.256499 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hskfk_a4adc67b-70b1-40cb-83f2-1d01aa96683e/kube-rbac-proxy/0.log" Jul 02 19:46:54.278794 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:54.278773 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hskfk_a4adc67b-70b1-40cb-83f2-1d01aa96683e/init-textfile/0.log" Jul 02 19:46:54.590537 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:54.590502 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-dcvwl_6be4e678-7208-42b4-bf96-b6fd4bd121c4/prometheus-operator/0.log" Jul 02 19:46:54.614891 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:54.614859 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-dcvwl_6be4e678-7208-42b4-bf96-b6fd4bd121c4/kube-rbac-proxy/0.log" Jul 02 19:46:54.680625 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:54.680597 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cdbd58d97-mvhrt_3ef1c4aa-0869-47bf-b586-659a4cc9b961/telemeter-client/0.log" Jul 02 19:46:54.703087 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:54.703039 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cdbd58d97-mvhrt_3ef1c4aa-0869-47bf-b586-659a4cc9b961/reload/0.log" Jul 02 19:46:54.725921 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:54.725877 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cdbd58d97-mvhrt_3ef1c4aa-0869-47bf-b586-659a4cc9b961/kube-rbac-proxy/0.log" Jul 02 19:46:56.623277 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.623238 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj"] Jul 02 19:46:56.623788 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.623611 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39604f1f-5697-4b2d-b796-e2961ea8be5d" containerName="main" Jul 02 19:46:56.623788 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.623623 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="39604f1f-5697-4b2d-b796-e2961ea8be5d" containerName="main" Jul 02 19:46:56.623788 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.623632 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d718c32c-88f5-45cc-9309-92624d5a0181" containerName="main" Jul 02 19:46:56.623788 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.623637 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d718c32c-88f5-45cc-9309-92624d5a0181" containerName="main" Jul 02 19:46:56.623788 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.623720 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d718c32c-88f5-45cc-9309-92624d5a0181" containerName="main" Jul 02 19:46:56.623788 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.623731 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="39604f1f-5697-4b2d-b796-e2961ea8be5d" containerName="main" Jul 02 19:46:56.626785 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.626765 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" Jul 02 19:46:56.628971 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.628953 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qqkdz\"/\"default-dockercfg-865ch\"" Jul 02 19:46:56.629122 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.629104 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qqkdz\"/\"kube-root-ca.crt\"" Jul 02 19:46:56.629853 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.629838 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qqkdz\"/\"openshift-service-ca.crt\"" Jul 02 19:46:56.636976 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.636954 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj"] Jul 02 19:46:56.698176 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.698139 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7055c84-7c40-4411-bcdf-00ea4918ccce-lib-modules\") pod \"perf-node-gather-daemonset-8rxmj\" (UID: \"a7055c84-7c40-4411-bcdf-00ea4918ccce\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" Jul 02 19:46:56.698333 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.698206 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7055c84-7c40-4411-bcdf-00ea4918ccce-sys\") pod \"perf-node-gather-daemonset-8rxmj\" (UID: \"a7055c84-7c40-4411-bcdf-00ea4918ccce\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" Jul 02 19:46:56.698333 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.698224 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8796\" (UniqueName: \"kubernetes.io/projected/a7055c84-7c40-4411-bcdf-00ea4918ccce-kube-api-access-g8796\") pod \"perf-node-gather-daemonset-8rxmj\" (UID: \"a7055c84-7c40-4411-bcdf-00ea4918ccce\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" Jul 02 19:46:56.698333 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.698268 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a7055c84-7c40-4411-bcdf-00ea4918ccce-proc\") pod \"perf-node-gather-daemonset-8rxmj\" (UID: \"a7055c84-7c40-4411-bcdf-00ea4918ccce\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" Jul 02 19:46:56.698333 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.698286 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a7055c84-7c40-4411-bcdf-00ea4918ccce-podres\") pod \"perf-node-gather-daemonset-8rxmj\" (UID: \"a7055c84-7c40-4411-bcdf-00ea4918ccce\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" Jul 02 19:46:56.799363 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.799333 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a7055c84-7c40-4411-bcdf-00ea4918ccce-proc\") pod \"perf-node-gather-daemonset-8rxmj\" (UID: \"a7055c84-7c40-4411-bcdf-00ea4918ccce\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" Jul 02 19:46:56.799363 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.799364 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a7055c84-7c40-4411-bcdf-00ea4918ccce-podres\") pod \"perf-node-gather-daemonset-8rxmj\" (UID: \"a7055c84-7c40-4411-bcdf-00ea4918ccce\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" Jul 02 19:46:56.799621 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.799409 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7055c84-7c40-4411-bcdf-00ea4918ccce-lib-modules\") pod \"perf-node-gather-daemonset-8rxmj\" (UID: \"a7055c84-7c40-4411-bcdf-00ea4918ccce\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" Jul 02 19:46:56.799621 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.799454 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7055c84-7c40-4411-bcdf-00ea4918ccce-sys\") pod \"perf-node-gather-daemonset-8rxmj\" (UID: \"a7055c84-7c40-4411-bcdf-00ea4918ccce\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" Jul 02 19:46:56.799621 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.799464 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a7055c84-7c40-4411-bcdf-00ea4918ccce-proc\") pod \"perf-node-gather-daemonset-8rxmj\" (UID: \"a7055c84-7c40-4411-bcdf-00ea4918ccce\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" Jul 02 19:46:56.799621 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.799470 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8796\" (UniqueName: \"kubernetes.io/projected/a7055c84-7c40-4411-bcdf-00ea4918ccce-kube-api-access-g8796\") pod \"perf-node-gather-daemonset-8rxmj\" (UID: \"a7055c84-7c40-4411-bcdf-00ea4918ccce\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" Jul 02 19:46:56.799621 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.799534 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a7055c84-7c40-4411-bcdf-00ea4918ccce-podres\") pod \"perf-node-gather-daemonset-8rxmj\" (UID: \"a7055c84-7c40-4411-bcdf-00ea4918ccce\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" Jul 02 19:46:56.799621 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.799559 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7055c84-7c40-4411-bcdf-00ea4918ccce-sys\") pod \"perf-node-gather-daemonset-8rxmj\" (UID: \"a7055c84-7c40-4411-bcdf-00ea4918ccce\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" Jul 02 19:46:56.799621 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.799586 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7055c84-7c40-4411-bcdf-00ea4918ccce-lib-modules\") pod \"perf-node-gather-daemonset-8rxmj\" (UID: \"a7055c84-7c40-4411-bcdf-00ea4918ccce\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" Jul 02 19:46:56.807463 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.807441 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8796\" (UniqueName: \"kubernetes.io/projected/a7055c84-7c40-4411-bcdf-00ea4918ccce-kube-api-access-g8796\") pod \"perf-node-gather-daemonset-8rxmj\" (UID: \"a7055c84-7c40-4411-bcdf-00ea4918ccce\") " pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" Jul 02 19:46:56.937860 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:56.937762 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" Jul 02 19:46:57.072734 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:57.072531 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj"] Jul 02 19:46:57.075467 ip-10-0-143-205 kubenswrapper[2566]: W0702 19:46:57.075439 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda7055c84_7c40_4411_bcdf_00ea4918ccce.slice/crio-6b1edc9912135fea4cd9b863820581fba06de340b7a5ab3d43523857d47be486 WatchSource:0}: Error finding container 6b1edc9912135fea4cd9b863820581fba06de340b7a5ab3d43523857d47be486: Status 404 returned error can't find the container with id 6b1edc9912135fea4cd9b863820581fba06de340b7a5ab3d43523857d47be486 Jul 02 19:46:57.155560 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:57.155541 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5854c674c6-czlnf_6a59376e-4d55-42d0-8c31-0bf26d144a9f/console/0.log" Jul 02 19:46:57.804820 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:57.804780 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" event={"ID":"a7055c84-7c40-4411-bcdf-00ea4918ccce","Type":"ContainerStarted","Data":"55bbd99d6887bc77f82e1c7b7050256d47da7531f637a3597be032311adff37f"} Jul 02 19:46:57.805229 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:57.804840 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" event={"ID":"a7055c84-7c40-4411-bcdf-00ea4918ccce","Type":"ContainerStarted","Data":"6b1edc9912135fea4cd9b863820581fba06de340b7a5ab3d43523857d47be486"} Jul 02 19:46:57.805229 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:57.804927 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" Jul 02 19:46:57.823732 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:57.823652 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" podStartSLOduration=1.823636987 podStartE2EDuration="1.823636987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 19:46:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 19:46:57.82317995 +0000 UTC m=+2497.750188662" watchObservedRunningTime="2026-07-02 19:46:57.823636987 +0000 UTC m=+2497.750645698" Jul 02 19:46:58.818618 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:58.818591 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-66wg5_3c79663f-9a62-40e4-b870-f2f156b654d0/dns/0.log" Jul 02 19:46:58.841876 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:58.841844 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-66wg5_3c79663f-9a62-40e4-b870-f2f156b654d0/kube-rbac-proxy/0.log" Jul 02 19:46:58.942580 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:58.942550 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hbf5k_a04cbb97-93a7-4d61-bdf9-deba1968e7c0/dns-node-resolver/0.log" Jul 02 19:46:59.575403 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:46:59.575374 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-45z57_d3b9eda7-ed8d-4a5b-9f6d-7c13e252fe5e/node-ca/0.log" Jul 02 19:47:00.685105 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:00.685071 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c665485c8-s2bzz_2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74/kube-auth-proxy/0.log" Jul 02 19:47:01.698789 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:01.698759 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s5ft7_7ac120d5-4418-44db-adea-9c186a1a46af/serve-healthcheck-canary/0.log" Jul 02 19:47:02.196459 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:02.196430 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d29gg_6d13f0e4-7378-429b-a212-431f984561db/kube-rbac-proxy/0.log" Jul 02 19:47:02.219332 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:02.219305 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d29gg_6d13f0e4-7378-429b-a212-431f984561db/exporter/0.log" Jul 02 19:47:02.243786 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:02.243761 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-d29gg_6d13f0e4-7378-429b-a212-431f984561db/extractor/0.log" Jul 02 19:47:03.820042 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:03.820009 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qqkdz/perf-node-gather-daemonset-8rxmj" Jul 02 19:47:05.282219 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:05.282189 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cf784d988-ghsw5_24127ebd-5316-48db-b150-026731d58a15/manager/0.log" Jul 02 19:47:05.301095 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:05.301073 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cf784d988-ghsw5_24127ebd-5316-48db-b150-026731d58a15/copy-manifests/0.log" Jul 02 19:47:06.874470 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:06.874387 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5556b8b6b-qb572_c997ebe2-d60c-4d2b-8e1a-e0d0908ea978/manager/0.log" Jul 02 19:47:06.898386 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:06.898352 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c665485c8-s2bzz_2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74/kube-auth-proxy/0.log" Jul 02 19:47:06.899434 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:06.899416 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-zns4z_b699ddd3-fa62-40c8-82a3-04e78d6ff522/openshift-lws-operator/0.log" Jul 02 19:47:08.450768 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:08.450665 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-c665485c8-s2bzz_2d64ee9e-1fd2-4c68-aa84-d23d8afc4b74/kube-auth-proxy/0.log" Jul 02 19:47:11.818733 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:11.818703 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-2kswt_b544ec3c-7ef0-4490-8c4f-87384346634f/migrator/0.log" Jul 02 19:47:11.841362 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:11.841333 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-2kswt_b544ec3c-7ef0-4490-8c4f-87384346634f/graceful-termination/0.log" Jul 02 19:47:13.222733 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:13.222705 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6nnrq_8e4ef969-3b7b-41a4-865b-f15c27d2cd69/kube-multus-additional-cni-plugins/0.log" Jul 02 19:47:13.245906 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:13.245881 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6nnrq_8e4ef969-3b7b-41a4-865b-f15c27d2cd69/egress-router-binary-copy/0.log" Jul 02 19:47:13.270984 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:13.270957 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6nnrq_8e4ef969-3b7b-41a4-865b-f15c27d2cd69/cni-plugins/0.log" Jul 02 19:47:13.295296 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:13.295267 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6nnrq_8e4ef969-3b7b-41a4-865b-f15c27d2cd69/bond-cni-plugin/0.log" Jul 02 19:47:13.318915 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:13.318887 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6nnrq_8e4ef969-3b7b-41a4-865b-f15c27d2cd69/routeoverride-cni/0.log" Jul 02 19:47:13.340779 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:13.340749 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6nnrq_8e4ef969-3b7b-41a4-865b-f15c27d2cd69/whereabouts-cni-bincopy/0.log" Jul 02 19:47:13.363518 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:13.363493 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6nnrq_8e4ef969-3b7b-41a4-865b-f15c27d2cd69/whereabouts-cni/0.log" Jul 02 19:47:13.759380 ip-10-0-143-205 kubenswrapper[2566]: I0702 19:47:13.759351 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-z8v4v_534a1f91-37b3-4ba4-93fc-f872ed2b668f/kube-multus/0.log"